51-100 of 10000 results (19ms)
2020-07-06 §
11:09 <marostegui> Compress InnoDB on db1107 T254462 [production]
11:09 <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: f4b5001: Add arxiv.org to commonswiki wgCopyUploadsDomains (T257036) (duration: 00m 56s) [production]
11:07 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1107 T254462', diff saved to https://phabricator.wikimedia.org/P11745 and previous config saved to /var/cache/conftool/dbconfig/20200706-110723-marostegui.json [production]
11:05 <marostegui@cumin1001> dbctl commit (dc=all): 'Repool db1076', diff saved to https://phabricator.wikimedia.org/P11744 and previous config saved to /var/cache/conftool/dbconfig/20200706-110544-marostegui.json [production]
11:05 <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: 3bc1b46: Remove "Create a book" link from sidebar on Finnish Wikipedia (T257073) (duration: 00m 56s) [production]
10:52 <jdrewniak@deploy1001> Synchronized portals: Wikimedia Portals Update: [[gerrit:609762| Bumping portals to master (609762)]] (duration: 00m 57s) [production]
10:51 <jdrewniak@deploy1001> Synchronized portals/wikipedia.org/assets: Wikimedia Portals Update: [[gerrit:609762| Bumping portals to master (609762)]] (duration: 00m 56s) [production]
10:31 <jmm@cumin2001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) [production]
10:28 <jmm@cumin2001> START - Cookbook sre.hosts.reboot-single [production]
10:28 <moritzm> rebooting idp1001 for kernel update [production]
09:35 <urbanecm@deploy1001> Synchronized private/PrivateSettings.php: Update T250887 mitigations (duration: 00m 58s) [production]
08:51 <XioNoX> cr1-codfw> request vmhost snapshot routing-engine both - T257153 [production]
08:44 <XioNoX> cr3-ulsfo> request vmhost snapshot - T257153 [production]
08:24 <kormat> restarting all mariadb instances on sanitarium hosts T256545 [production]
08:09 <elukey> roll restart aqs on aqs100[4-9] to pick up new druid settings [production]
08:05 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1076', diff saved to https://phabricator.wikimedia.org/P11742 and previous config saved to /var/cache/conftool/dbconfig/20200706-080509-marostegui.json [production]
07:58 <qchris> Disable puppet on gerrit1002 (gerrit-test) to deploy Gerrit UI updates there to gather more feedback [production]
07:51 <elukey> enable binlog on matomo's database on matomo1002 [production]
07:46 <XioNoX> repool eqsin - T257154 [production]
07:11 <XioNoX> reboot cr3-eqsin - T257154 [production]
06:55 <XioNoX> depool eqsin for cr3-eqsin reboot/investigation - T257154 [production]
06:54 <marostegui@cumin1001> dbctl commit (dc=all): 'Fully repool db1089', diff saved to https://phabricator.wikimedia.org/P11740 and previous config saved to /var/cache/conftool/dbconfig/20200706-065437-marostegui.json [production]
06:54 <elukey@cumin1001> END (FAIL) - Cookbook sre.hadoop.change-distro (exit_code=99) [production]
06:22 <elukey@cumin1001> START - Cookbook sre.hadoop.change-distro [production]
06:21 <elukey@cumin1001> END (PASS) - Cookbook sre.hadoop.stop-cluster (exit_code=0) [production]
06:14 <elukey@cumin1001> START - Cookbook sre.hadoop.stop-cluster [production]
05:45 <kart_> Updated cxserver to 2020-07-01-044435-production (T254143) [production]
05:40 <kartik@deploy1001> helmfile [CODFW] Ran 'sync' command on namespace 'cxserver' for release 'production' . [production]
05:36 <kartik@deploy1001> helmfile [EQIAD] Ran 'sync' command on namespace 'cxserver' for release 'production' . [production]
05:32 <kartik@deploy1001> helmfile [STAGING] Ran 'sync' command on namespace 'cxserver' for release 'staging' . [production]
05:13 <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1089', diff saved to https://phabricator.wikimedia.org/P11739 and previous config saved to /var/cache/conftool/dbconfig/20200706-051333-marostegui.json [production]
05:03 <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1089', diff saved to https://phabricator.wikimedia.org/P11738 and previous config saved to /var/cache/conftool/dbconfig/20200706-050347-marostegui.json [production]
04:49 <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1089', diff saved to https://phabricator.wikimedia.org/P11737 and previous config saved to /var/cache/conftool/dbconfig/20200706-044908-marostegui.json [production]
2020-07-05 §
21:50 <qchris> Restarting gerrit on gerrit1001 to pick up new war and jars. [production]
21:50 <qchris@deploy1001> Finished deploy [gerrit/gerrit@fbd0684]: Bump gerrit to 3.2.2-102-g3bbb138e13, zuul plugin to master-0-g7accc67, and gitiles to v3.2.2-1-g00c5ca0-with-0e3b533 on gerrit1001 (duration: 00m 07s) [production]
21:50 <qchris@deploy1001> Started deploy [gerrit/gerrit@fbd0684]: Bump gerrit to 3.2.2-102-g3bbb138e13, zuul plugin to master-0-g7accc67, and gitiles to v3.2.2-1-g00c5ca0-with-0e3b533 on gerrit1001 [production]
21:46 <qchris> Restarting gerrit on gerrit2001 to pick up new war and jars. [production]
21:45 <qchris@deploy1001> Finished deploy [gerrit/gerrit@fbd0684]: Bump gerrit to 3.2.2-102-g3bbb138e13, zuul plugin to master-0-g7accc67, and gitiles to v3.2.2-1-g00c5ca0-with-0e3b533 on gerrit2001 (duration: 00m 10s) [production]
21:45 <qchris@deploy1001> Started deploy [gerrit/gerrit@fbd0684]: Bump gerrit to 3.2.2-102-g3bbb138e13, zuul plugin to master-0-g7accc67, and gitiles to v3.2.2-1-g00c5ca0-with-0e3b533 on gerrit2001 [production]
21:32 <qchris> Restarting gerrit on gerrit1002 to pick up new wars and jars. [production]
21:32 <qchris@deploy1001> Finished deploy [gerrit/gerrit@fbd0684]: Bump gerrit to 3.2.2-102-g3bbb138e13 and zuul plugin to master-0-g7accc67 (duration: 00m 08s) [production]
21:32 <qchris@deploy1001> Started deploy [gerrit/gerrit@fbd0684]: Bump gerrit to 3.2.2-102-g3bbb138e13 and zuul plugin to master-0-g7accc67 [production]
21:20 <qchris> Enable puppet on gerrit1002 (gerrit-test) again to let it catch up again [production]
16:01 <gehel> restart elastic-psi on elastic1052 (high GC rate) [production]
15:56 <gehel> restart blazegraph + updater on wdqs1007 and depool to allow catching up on lag [production]
2020-07-04 §
19:23 <qchris@deploy1001> Finished deploy [gerrit/gerrit@b78914b]: Bump gitiles to v3.2.2-1-g00c5ca0-with-0e3b533 on gerrit1002 (duration: 00m 08s) [production]
19:23 <qchris@deploy1001> Started deploy [gerrit/gerrit@b78914b]: Bump gitiles to v3.2.2-1-g00c5ca0-with-0e3b533 on gerrit1002 [production]
14:05 <qchris> Disable puppet on gerrit1002 (gerrit-test) to deploy Gerrit UI updates there to gather feedback [production]
12:42 <reedy@deploy1001> Synchronized wmf-config/interwiki.php: Update interwiki cache (duration: 02m 24s) [production]
02:28 <reedy@deploy1001> Synchronized php-1.35.0-wmf.39/extensions/Score/includes/Score.php: Short circuit lilypond version check to allow usage of cached files T257066 (duration: 00m 55s) [production]