51-100 of 10000 results (50ms)
2018-10-11 §
14:36 <END> (PASS) - Cookbook sre.switchdc.services.01-switch-dc (exit_code=0) (volans@neodymium) [production]
14:36 <Switching> services parsoid, restbase, restbase-async, mobileapps, apertium, citoid, cxserver, eventstreams, graphoid, mathoid, proton, pdfrender, recommendation-api, zotero, eventbus, ores, wdqs, wdqs-internal: codfw => eqiad (volans@neodymium) [production]
14:36 <START> - Cookbook sre.switchdc.services.01-switch-dc (volans@neodymium) [production]
14:35 <END> (PASS) - Cookbook sre.switchdc.services.00-reduce-ttl-and-sleep (exit_code=0) (volans@neodymium) [production]
14:30 <banyek@deploy1001> Synchronized wmf-config/db-eqiad.php: T206743: mariadb: Depool db1087 (duration: 00m 49s) [production]
14:30 <START> - Cookbook sre.switchdc.services.00-reduce-ttl-and-sleep (volans@neodymium) [production]
14:28 <banyek> depooling db1087 (T206743) [production]
14:28 <banyek> depooling db1087 [production]
14:15 <elukey> reboot eventlog1002 for kernel upgrades [production]
14:15 <jynus> applying row filling to (most) eqiad s8 dbs, including the mater [production]
14:13 <moritzm> install libxml2 security updates on jessie servers [production]
13:55 <jynus> recovering rows to db1092 [production]
13:26 <jynus> filling in missing rows on dbstore1002 [production]
13:23 <marostegui> Stop MySQL on db2083 to reclone db1116:3318 - T206743 [production]
13:21 <marostegui@deploy1001> Synchronized wmf-config/db-codfw.php: Depool db2083 (duration: 00m 49s) [production]
13:20 <marostegui> Stop MySQL on db1116:3318 to reclone it from db2083 - T206743 [production]
12:43 <elukey> upgrade prometheus-memcached-exporter on mc1* [production]
12:38 <elukey> upgrade prometheus-memcached-exporter on mc2* [production]
12:15 <elukey> upgrade prometheus-memcached-exporter on mc2035 [production]
12:14 <elukey> upload prometheus-memcached-exporter_0.4.1+git20181010.2fa99eb-1 to (jessie|stretch)-wikimedia [production]
12:12 <Amir1> EU SWAT is done [production]
11:30 <ladsgroup@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:466271|Set some small wikis to read new for change tag backend (T194164)]] (duration: 00m 50s) [production]
11:10 <marostegui> Stop MYSQL on db2085:3318 and db1099:3318 T206743 [production]
11:09 <marostegui> Stop MYSQL on db2088:3318 and db1099:3318 T206743 [production]
11:08 <marostegui@deploy1001> Synchronized wmf-config/db-codfw.php: Depool db2085:3318 and db1099:3318 (duration: 00m 49s) [production]
11:07 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Depool db2085:3318 and db1099:3318 (duration: 00m 49s) [production]
11:07 <banyek> binlog expiration set to 60 days on db2045 [production]
08:30 <banyek> setting up some automated binlog purge mechanism on pc1004,pc1005,pc1006 [production]
08:26 <jynus> setting up replication from pc2005 -> pc1005 and from pc2006 -> pc2006 [production]
08:20 <jynus> setting up replication from pc2004 -> pc1004 [production]
08:04 <banyek> purging binary logs on pc1006 [production]
08:04 <banyek> purging binary logs on pc1005 [production]
08:04 <jynus> running /usr/local/bin/mwscript purgeParserCache.php --wiki=aawiki --age=1900800 --msleep 0 [production]
08:04 <banyek> purging binary logs on pc1004 [production]
07:57 <gehel> rolling restart blazegraph on wdqs-internal for config change - T206648 [production]
07:43 <addshore> deploy https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/Wikibase/+/466031 to mwmaint1002 only (increasing tracking of wikidata dispatching) T205865 [production]
07:36 <elukey> roll restart of aqs on aqs100[4-9] to pick up new Druid settings [production]
06:00 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Increase db1092 weight (duration: 00m 49s) [production]
05:43 <marostegui> Purge binary logs on pc2005 due to disk space issues - T206740 [production]
05:29 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Repool db1109 (duration: 00m 48s) [production]
05:24 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Depool db1109 (duration: 00m 51s) [production]
02:25 <krinkle@deploy1001> Synchronized w/static.php: T127233 - Ic6acb70 (duration: 00m 49s) [production]
02:10 <krinkle@deploy1001> Synchronized php-1.32.0-wmf.24/includes/page/WikiPage.php: T203942 - Ib211d98498f (duration: 00m 49s) [production]
02:07 <krinkle@deploy1001> Synchronized php-1.32.0-wmf.24/tests/phpunit/includes/page/: Ib211d98498f (duration: 00m 49s) [production]
01:38 <krinkle@deploy1001> Synchronized wmf-config/etcd.php: T176370 - I5e7e5d167d517 (duration: 00m 55s) [production]
2018-10-10 §
23:08 <krinkle@deploy1001> Synchronized php-1.32.0-wmf.24/maintenance/resources/foreign-resources.yaml: Ic865e7077d (duration: 00m 49s) [production]
22:59 <krinkle@deploy1001> Synchronized php-1.32.0-wmf.24/extensions/MultimediaViewer/: T206099 - I53dbce0a (duration: 00m 49s) [production]
22:43 <krinkle@deploy1001> Synchronized php-1.32.0-wmf.24/includes/specials/SpecialDeletedContributions.php: T187619 - Ic6b0d8020553 (duration: 00m 48s) [production]
22:41 <krinkle@deploy1001> Synchronized php-1.32.0-wmf.24/extensions/ORES/includes/FetchScoreJob.php: T204753 - Icc28230585bc (duration: 00m 49s) [production]
22:25 <mutante> icinga1001 - chmod 2710 /var/lib/icinga/rw [production]