3251-3300 of 10000 results (65ms)
2019-11-11 §
13:25 <ema> depool cp3050 and reimage as text_ats T227432 [production]
12:59 <gehel@cumin1001> START - Cookbook sre.wdqs.data-reload [production]
12:46 <effie> Upgrade to 7.2.24-1 mwdebug[2001-2002].codfw.wmnet,mwmaint2001.codfw.wmnet,deploy2001.codfw.wmnet - T237239 [production]
12:31 <onimisionipe@deploy1001> Finished deploy [wdqs/wdqs@2cb2dde]: Deploy updates on wdqs1010 (duration: 00m 28s) [production]
12:30 <onimisionipe@deploy1001> Started deploy [wdqs/wdqs@2cb2dde]: Deploy updates on wdqs1010 [production]
12:28 <effie> Upgrade mw2* to 7.2.24-1 with elegance and restart php-fpm - T237239 [production]
12:21 <effie> Upgrade mw2* to 7.2.24-1 with elegance and restart php-fpm - T231881 [production]
11:55 <gehel@cumin1001> END (FAIL) - Cookbook sre.wdqs.data-reload (exit_code=99) [production]
10:52 <hoo> Updated the Wikidata property suggester with data from the 2019-11-04 JSON dump and applied the T132839 workarounds [production]
10:48 <gehel@cumin1001> START - Cookbook sre.wdqs.data-reload [production]
10:47 <gehel@cumin1001> END (FAIL) - Cookbook sre.wdqs.data-reload (exit_code=99) [production]
10:45 <gehel@cumin1001> START - Cookbook sre.wdqs.data-reload [production]
10:32 <vgutierrez> restarting ats-tls on cp1088 [production]
10:21 <jynus> upgrade mariadb on db2102 [production]
10:16 <ema> repool cp4027 after successful X-Wikimedia-Debug testing P9585 T237687 [production]
10:12 <jynus> manually run full backup of labtestpuppetmaster2001 T235819 [production]
09:41 <ema> test x-wikimedia-debug-routing.lua on cp4027 (depooled) T237687 [production]
09:09 <volker-e@deploy1001> Finished deploy [design/style-guide@0ea65f2]: Deploy design/style-guide: (duration: 00m 07s) [production]
09:09 <volker-e@deploy1001> Started deploy [design/style-guide@0ea65f2]: Deploy design/style-guide: [production]
08:28 <marostegui> Stop MySQL on db2048 before decommissioning - T237913 [production]
08:28 <marostegui@deploy1001> Synchronized wmf-config/db-codfw.php: Remove db2048 from config T237913 (duration: 00m 51s) [production]
08:27 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Remove db2048 from config T237913 (duration: 00m 54s) [production]
08:21 <marostegui> Remove db2048 from tendril and zarcillo T237913 [production]
06:56 <elukey> delete /etc/logrotate.d/wdqs-reload-categories from wdqs* as attempt to reduce cronspam [production]
06:44 <marostegui> Delete globalblocks table from napwikisource T230055 [production]
05:27 <vgutierrez> Switch from nginx to ats-tls on cp3058 - T231627 [production]
2019-11-09 §
20:25 <reedy@deploy1001> Synchronized langlist-labs: T237823 (duration: 00m 54s) [production]
02:39 <volker-e@deploy1001> Finished deploy [design/style-guide@d2bfc09]: Deploy design/style-guide: (duration: 00m 07s) [production]
02:39 <volker-e@deploy1001> Started deploy [design/style-guide@d2bfc09]: Deploy design/style-guide: [production]
01:07 <volker-e@deploy1001> Finished deploy [design/style-guide@ef82b69]: Deploy design/style-guide: (duration: 00m 07s) [production]
01:07 <volker-e@deploy1001> Started deploy [design/style-guide@ef82b69]: Deploy design/style-guide: [production]
01:06 <volker-e@deploy1001> Finished deploy [design/style-guide@97fb3ee]: Deploy design/style-guide: (duration: 00m 09s) [production]
01:06 <volker-e@deploy1001> Started deploy [design/style-guide@97fb3ee]: Deploy design/style-guide: [production]
2019-11-08 §
20:26 <mholloway-shell@deploy1001> Synchronized wmf-config/InitialiseSettings.php: MachineVision: Delay annotation request jobs by 5 mins for testing (duration: 00m 52s) [production]
16:54 <jeh@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
16:52 <jeh@cumin1001> START - Cookbook sre.hosts.downtime [production]
16:19 <jeh@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
16:15 <jeh@cumin1001> START - Cookbook sre.hosts.downtime [production]
16:15 <mholloway-shell@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Revert "MachineVision: Enable testers-only mode on testcommonswiki for debugging" (duration: 00m 54s) [production]
15:57 <jynus@cumin1001> dbctl commit (dc=all): 'Pool db1118, db1106 at 100%', diff saved to https://phabricator.wikimedia.org/P9582 and previous config saved to /var/cache/conftool/dbconfig/20191108-155700-jynus.json [production]
15:37 <herron> beginning rolling service restarts on logstash hosts for java security updates [production]
15:13 <mholloway-shell@deploy1001> Synchronized wmf-config/InitialiseSettings.php: MachineVision: Enable testers-only mode on testcommonswiki for debugging (duration: 00m 52s) [production]
14:56 <volans@cumin1001> END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) [production]
14:55 <volans@cumin1001> START - Cookbook sre.hosts.decommission [production]
14:50 <jynus@cumin1001> dbctl commit (dc=all): 'Pool db1118 at 50%', diff saved to https://phabricator.wikimedia.org/P9581 and previous config saved to /var/cache/conftool/dbconfig/20191108-145028-jynus.json [production]
14:42 <volans@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
14:40 <volans@cumin1001> START - Cookbook sre.hosts.downtime [production]
13:40 <jynus> stop and upgrade percona-server on test host db1114 [production]
13:27 <elukey@cumin1001> END (PASS) - Cookbook sre.cassandra.roll-restart (exit_code=0) [production]
13:12 <jynus@cumin1001> dbctl commit (dc=all): 'Pool db1118 at 20%', diff saved to https://phabricator.wikimedia.org/P9580 and previous config saved to /var/cache/conftool/dbconfig/20191108-131257-jynus.json [production]