1751-1800 of 10000 results (54ms)
2018-12-18 §
09:38 <godog> swift eqiad-prod: initial weights for ms-be10[44-50].eqiad.wmnet - T209618 [production]
08:44 <marostegui> Enable GTID on s7 codfw master (db2040) - T211973 [production]
08:26 <marostegui> Enable GTID on es3 - T211973 [production]
08:23 <marostegui> Enable GTID on es2 - T211973 [production]
07:57 <elukey> restart cassandra-{a,b} on aqs1004 for openjdk upgrades [production]
07:49 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Repool db1078 T86338 T202167 (duration: 00m 45s) [production]
07:49 <marostegui> Deploy schema change on db1075 (s3 master) T86338 T202167 [production]
06:07 <marostegui> Deploy schema change on db1078 T86338 T202167 [production]
06:07 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Depool db1078 T86338 T202167 (duration: 00m 45s) [production]
06:03 <marostegui@deploy1001> Synchronized wmf-config/db-codfw.php: Repool db2038 after mysql and kernel upgrade (duration: 00m 47s) [production]
00:13 <tgr@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:479997|Disable MediaViewer thumbnail URL guessing for private wikis (T212099)]] (duration: 00m 45s) [production]
2018-12-17 §
23:21 <tgr> deployed security patch per T207750#4829275 [production]
23:17 <chasemp> launch gpg jobs for files on stat1004 [production]
23:17 <chasemp> launch gpg jobs for files on labstore1007 [production]
22:00 <mutante> puppetmaster - signed cert for doc1001 (ganeti VM), initial puppet run [production]
21:58 <chasemp> stat1004:/var/log# mkfs.exfat /dev/sde && mkdir /mnt/T211327 && mount /dev/sde /mnt/T211327/ [production]
21:55 <arlolra> Updated Parsoid to 4eba44e (T204622, T211941) [production]
21:42 <arlolra@deploy1001> Finished deploy [parsoid/deploy@1bf4dab]: Updating Parsoid to 4eba44e (duration: 08m 55s) [production]
21:33 <arlolra@deploy1001> Started deploy [parsoid/deploy@1bf4dab]: Updating Parsoid to 4eba44e [production]
21:20 <mholloway-shell@deploy1001> Finished deploy [mobileapps/deploy@92fdf43]: Update mobileapps to d244439 (duration: 04m 31s) [production]
21:15 <mholloway-shell@deploy1001> Started deploy [mobileapps/deploy@92fdf43]: Update mobileapps to d244439 [production]
20:25 <ppchelko@deploy1001> Finished deploy [recommendation-api/deploy@c1b6b32]: Rollback to c1b6b324015bd565 until the checks are fixed (duration: 01m 56s) [production]
20:23 <ppchelko@deploy1001> Started deploy [recommendation-api/deploy@c1b6b32]: Rollback to c1b6b324015bd565 until the checks are fixed [production]
20:17 <mutante> creating new ganeti VM doc1001.eqiad.wmnet for doc.wikimedia.org - specs as requested by hashar on T211974 [production]
20:05 <XioNoX> remove sandbox-out4 from all routers - T212155 [production]
20:02 <XioNoX> remove sandbox-out4 from ulsfo - T212155 [production]
19:47 <ppchelko@deploy1001> Finished deploy [recommendation-api/deploy@f183af7]: Update to 657a515. All hosts (duration: 10m 45s) [production]
19:36 <ppchelko@deploy1001> Started deploy [recommendation-api/deploy@f183af7]: Update to 657a515. All hosts [production]
19:33 <ppchelko@deploy1001> Finished deploy [recommendation-api/deploy@f183af7]: Update to 657a515. Canary on scb2001 (duration: 00m 18s) [production]
19:32 <ppchelko@deploy1001> Started deploy [recommendation-api/deploy@f183af7]: Update to 657a515. Canary on scb2001 [production]
19:29 <raynor> Morning SWAT finished [production]
19:27 <pmiazga@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:480127 | Page issues treatment enabled on all wikis except enwiki(T210553)]] (duration: 00m 45s) [production]
18:58 <ppchelko@deploy1001> Finished deploy [recommendation-api/deploy@8036f9b]: Update to 2991db1. Canary on scb2001 (duration: 00m 51s) [production]
18:57 <ppchelko@deploy1001> Started deploy [recommendation-api/deploy@8036f9b]: Update to 2991db1. Canary on scb2001 [production]
18:42 <herron> manually restarting pybal on lvs2003 to add kibana service T205850 [production]
18:34 <herron@puppetmaster1001> conftool action : set/pooled=yes; selector: dc=codfw,cluster=logstash,service=kibana,name=logstash2006.codfw.wmnet [production]
18:34 <herron@puppetmaster1001> conftool action : set/pooled=yes; selector: dc=codfw,cluster=logstash,service=kibana,name=logstash2005.codfw.wmnet [production]
18:34 <mobrovac@deploy1001> Finished deploy [proton/deploy@ff7c8a2]: Config: Use extdomain and add printable=yes to the req query - T210793 (duration: 01m 32s) [production]
18:32 <herron> performing manual puppet run and manually restarting pybal on lvs2006 to add kibana service T205850 [production]
18:32 <mobrovac@deploy1001> Started deploy [proton/deploy@ff7c8a2]: Config: Use extdomain and add printable=yes to the req query - T210793 [production]
18:29 <gtirloni> prometheus-node-exporter: Ignore Docker/Kubelet mount points T211810 [production]
18:26 <akosiaris@deploy1001> scap-helm blubberoid finished [production]
18:26 <akosiaris@deploy1001> scap-helm blubberoid cluster codfw completed [production]
18:26 <akosiaris@deploy1001> scap-helm blubberoid install --name production --set docker.registry=docker-registry.discovery.wmnet --set main_app.version=2018-12-13-183249-production --set service.deployment=production --set service.externalIP=10.2.2.31 --set service.port=8748 stable/blubberoid [namespace: blubberoid, clusters: codfw] [production]
18:25 <akosiaris@deploy1001> scap-helm blubberoid finished [production]
18:25 <akosiaris@deploy1001> scap-helm blubberoid cluster eqiad completed [production]
18:25 <akosiaris@deploy1001> scap-helm blubberoid install --name production --set docker.registry=docker-registry.discovery.wmnet --set main_app.version=2018-12-13-183249-production --set service.deployment=production --set service.externalIP=10.2.1.31 --set service.port=8748 stable/blubberoid [namespace: blubberoid, clusters: eqiad] [production]
18:24 <akosiaris@deploy1001> scap-helm blubberoid install --name production --set docker.registry=docker-registry.discovery.wmnet --set main_app.version=2018-12-13-183249-production --set service.deployment=production --set service.externalIP=10.2.1.31 --set service.port=8748 stable/blubberoid [namespace: blubberoid, clusters: eqiad] [production]
18:23 <akosiaris@deploy1001> scap-helm blubberoid install --name production --set docker.registry=docker-registry.discovery.wmnet --set main_app.version=2018-12-13-183249-production --set service.deployment=production --set service.port=8748 stable/blubberoid [namespace: blubberoid, clusters: eqiad,codfw] [production]
18:21 <onimisionipe@deploy1001> Finished deploy [wdqs/wdqs@69c000a]: DelayQueue fix and mitigation of stale index for updater (duration: 11m 52s) [production]