1601-1650 of 10000 results (49ms)
2019-07-09 §
11:29 <jbond@cumin1001> START - Cookbook sre.hosts.downtime [production]
11:26 <jbond@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
11:26 <jbond@cumin1001> START - Cookbook sre.hosts.downtime [production]
11:13 <Urbanecm> EU SWAT done [production]
11:12 <urbanecm@deploy1001> Synchronized wmf-config/flaggedrevs.php: SWAT: [[:gerrit:521383|Disable flaggedrevs for hewikisource main page]] (T227000) (duration: 00m 48s) [production]
11:11 <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[:gerrit:521390|Clean up `wgNamespacesWithSubpages` to remove unneeded entries]] (T227546) (duration: 00m 49s) [production]
11:09 <urbanecm@deploy1001> Synchronized wmf-config/CommonSettings.php: SWAT: [[:gerrit:517933|Configuration migration for Translate]] (T87985) (duration: 00m 49s) [production]
11:04 <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[:gerrit:521298|Configure help urls for MediaInfo]] (T227226) (duration: 00m 50s) [production]
10:39 <elukey> update wikimedia-buster thirparty/amd-rocm component with upstream packages - T224723 [production]
10:14 <jbond42> upgrade openssl on canary systems [production]
09:30 <ema@puppetmaster1001> conftool action : set/pooled=yes; selector: name=cp1076.eqiad.wmnet,service=ats-be [production]
09:26 <ema> cp1076: restart trafficserver with storage.config set to /dev/nvme0n1 [production]
09:25 <ema@puppetmaster1001> conftool action : set/pooled=no; selector: name=cp1076.eqiad.wmnet,service=ats-be [production]
09:13 <elukey> enable per-server metrics on all prometheus-mcrouter-exporter(s) via puppet - T225059 [production]
09:11 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Fully repool db1086 after upgrade (duration: 00m 49s) [production]
08:56 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Slowly repool db1086 after upgrade (duration: 00m 47s) [production]
08:49 <elukey> upgrade prometheus-mcrouter-exporter to 0.0.0+git20190709-1 on mw-eqiad (cumin alias) via debdeploy - T225059 [production]
08:41 <marostegui> Upgrade db1086 [production]
08:41 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Depool db1086 for upgrade (duration: 00m 51s) [production]
08:36 <elukey> upgrade prometheus-mcrouter-exporter to 0.0.0+git20190709-1 on mw-codfw (cumin alias) via debdeploy - T225059 [production]
08:08 <moritzm> installing zeromq3 security updates [production]
08:00 <marostegui> Upgrade db1065 to 10.1.39 [production]
07:39 <moritzm> pruning unused libzmq3/python-zmq packages from swift/parsoid hosts [production]
07:26 <elukey> upload prometheus-mcrouter-exporter 0.0.0+git20190709-1 to stretch-wikimedia - T225059 [production]
06:00 <marostegui> Failover m2 from db1065 to db1132 - T226952 [production]
05:19 <marostegui> Start switchover steps T226952 [production]
05:13 <marostegui> Rebooting pc2010 for a second time as per papaul's suggestion T227552 [production]
05:13 <marostegui> Rebooting pc2010 for a second time as per papaul's suggestion T226952 [production]
04:53 <marostegui> Reboot pc2010 to debug a memory issue [production]
01:47 <XioNoX> restart PHP FPM on mwdebug2001 [production]
01:35 <XioNoX> restart PHP FPM on mwdebug1002 [production]
2019-07-08 §
23:03 <tzatziki> changing password for user "Naomi.piquette" [production]
20:57 <bd808> Upgraded prometheus-pdns-exporter to 0.4.1 on cloudservices1004.wikimedia.org (T227411) [production]
20:53 <bd808> Upgraded prometheus-pdns-exporter to 0.4.1 on cloudservices1003.wikimedia.org (T227411) [production]
19:38 <reedy@deploy1001> Synchronized php-1.34.0-wmf.11/extensions/OATHAuth/src/Key/TOTPKey.php: T227502 (duration: 00m 50s) [production]
19:23 <moritzm> uploaded prometheus-pdns-exporter 0.4.1 to stretch-wikimedia T227411 [production]
18:43 <otto@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Produce page-* streams to eventgate-main - T211248 (duration: 00m 50s) [production]
18:33 <moritzm> installing zeromq3 security updates [production]
18:15 <Urbanecm> Morning SWAT done [production]
18:14 <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[:gerrit:521308|Change liwikinews logo to correct one per community wish]] (2/2, T227418) (duration: 00m 49s) [production]
18:13 <urbanecm@deploy1001> Synchronized static/images/project-logos/: SWAT: [[:gerrit:521308|Change liwikinews logo to correct one per community wish]] (1/2, T227418) (duration: 00m 49s) [production]
18:10 <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[:gerrit:521191|Add templateeditor user group and protection level on commons]] (T227420) (duration: 00m 49s) [production]
18:06 <urbanecm@deploy1001> Synchronized wmf-config/CirrusSearch-production.php: SWAT: [[:gerrit:520446|[cirrus] Increase elastic master timeout to 5m]] (T227136) (duration: 00m 49s) [production]
18:04 <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[:gerrit:520078|Enable RDF output for MediaInfo]] (T221916) (duration: 00m 49s) [production]
17:20 <gehel@deploy1001> Finished deploy [wdqs/wdqs@4b7cdf5]: new blazegraph and updater version (duration: 12m 47s) [production]
17:08 <gehel@deploy1001> Started deploy [wdqs/wdqs@4b7cdf5]: new blazegraph and updater version [production]
16:40 <eevans@deploy1001> scap-helm sessionstore finished [production]
16:40 <eevans@deploy1001> scap-helm sessionstore cluster staging completed [production]
16:40 <eevans@deploy1001> scap-helm sessionstore upgrade staging -f sessionstore-staging-values.yaml stable/kask [namespace: sessionstore, clusters: staging] [production]
16:39 <eevans@deploy1001> scap-helm sessionstore finished [production]