5951-6000 of 10000 results (39ms)
2020-09-02 §
06:29 <oblivian@deploy1001> helmfile [eqiad] Ran 'sync' command on namespace 'termbox' for release 'test' . [production]
06:29 <oblivian@deploy1001> helmfile [eqiad] Ran 'sync' command on namespace 'termbox' for release 'staging' . [production]
06:21 <oblivian@deploy1001> helmfile [staging] Ran 'sync' command on namespace 'termbox' for release 'staging' . [production]
06:21 <oblivian@deploy1001> helmfile [staging] Ran 'sync' command on namespace 'termbox' for release 'test' . [production]
06:21 <oblivian@deploy1001> helmfile [staging] Ran 'sync' command on namespace 'termbox' for release 'production' . [production]
2020-09-01 §
22:39 <Urbanecm> [urbanecm@mwmaint2001 ~]$ mwscript extensions/OATHAuth/maintenance/disableOATHAuthForUser.php --wiki=sysop_itwiki Pierpao (T261722) [production]
17:51 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) [production]
17:50 <dzahn@cumin1001> START - Cookbook sre.hosts.decommission [production]
17:36 <ryankemper> wdqs [canary] rollback complete, tests passing now. Will need to dig into source of failure [production]
17:35 <ryankemper@deploy1001> Finished deploy [wdqs/wdqs@7920fbe]: 0.3.46 (duration: 03m 43s) [production]
17:35 <ryankemper> `wdqs1003` (the canary instance) is failing tests now, going to rollback [production]
17:32 <ryankemper@deploy1001> Started deploy [wdqs/wdqs@7920fbe]: 0.3.46 [production]
17:30 <ryankemper> Starting wdqs deploy [production]
15:56 <chasemp> labsdb* puppet agent --test; sudo /usr/local/sbin/maintain-views --all-databases --table user --replace-all; sudo /usr/local/sbin/maintain-views --all-databases --table user_old --replace-all [production]
15:25 <pt1979@cumin2001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
15:15 <pt1979@cumin2001> START - Cookbook sre.dns.netbox [production]
14:28 <_joe_> restarting envoy on all eqiad jobrunners [production]
14:22 <_joe_> restarted confd on mwmaint1002 [production]
14:18 <rzl@cumin1001> END (PASS) - Cookbook sre.switchdc.mediawiki.08-update-tendril (exit_code=0) [production]
14:18 <rzl@cumin1001> START - Cookbook sre.switchdc.mediawiki.08-update-tendril [production]
14:17 <rzl@cumin1001> END (PASS) - Cookbook sre.switchdc.mediawiki.08-start-maintenance (exit_code=0) [production]
14:15 <rzl@cumin1001> START - Cookbook sre.switchdc.mediawiki.08-start-maintenance [production]
14:15 <marostegui@cumin1001> dbctl commit (dc=all): 'Reduce db2083 weight', diff saved to https://phabricator.wikimedia.org/P12429 and previous config saved to /var/cache/conftool/dbconfig/20200901-141521-marostegui.json [production]
14:15 <rzl@cumin1001> END (PASS) - Cookbook sre.switchdc.mediawiki.08-restore-ttl (exit_code=0) [production]
14:14 <rzl@cumin1001> START - Cookbook sre.switchdc.mediawiki.08-restore-ttl [production]
14:07 <rzl@cumin1001> END (PASS) - Cookbook sre.switchdc.mediawiki.07-set-readwrite (exit_code=0) [production]
14:07 <rzl@cumin1001> MediaWiki read-only period ends at: 2020-09-01 14:07:36.305500 [production]
14:07 <rzl@cumin1001> START - Cookbook sre.switchdc.mediawiki.07-set-readwrite [production]
14:04 <rzl@cumin1001> END (FAIL) - Cookbook sre.switchdc.mediawiki.07-set-readwrite (exit_code=99) [production]
14:04 <rzl@cumin1001> START - Cookbook sre.switchdc.mediawiki.07-set-readwrite [production]
14:04 <rzl@cumin1001> END (PASS) - Cookbook sre.switchdc.mediawiki.06-set-db-readwrite (exit_code=0) [production]
14:04 <rzl@cumin1001> START - Cookbook sre.switchdc.mediawiki.06-set-db-readwrite [production]
14:04 <rzl@cumin1001> END (PASS) - Cookbook sre.switchdc.mediawiki.05-invert-redis-sessions (exit_code=0) [production]
14:04 <rzl@cumin1001> START - Cookbook sre.switchdc.mediawiki.05-invert-redis-sessions [production]
14:04 <rzl@cumin1001> END (PASS) - Cookbook sre.switchdc.mediawiki.04-switch-mediawiki (exit_code=0) [production]
14:03 <rzl@cumin1001> START - Cookbook sre.switchdc.mediawiki.04-switch-mediawiki [production]
14:03 <rzl@cumin1001> END (PASS) - Cookbook sre.switchdc.mediawiki.03-set-db-readonly (exit_code=0) [production]
14:02 <rzl@cumin1001> START - Cookbook sre.switchdc.mediawiki.03-set-db-readonly [production]
14:02 <rzl@cumin1001> END (PASS) - Cookbook sre.switchdc.mediawiki.02-set-readonly (exit_code=0) [production]
14:02 <rzl@cumin1001> MediaWiki read-only period starts at: 2020-09-01 14:02:04.851006 [production]
14:02 <rzl@cumin1001> START - Cookbook sre.switchdc.mediawiki.02-set-readonly [production]
13:58 <rzl@cumin1001> END (PASS) - Cookbook sre.switchdc.mediawiki.01-stop-maintenance (exit_code=0) [production]
13:58 <rzl@cumin1001> START - Cookbook sre.switchdc.mediawiki.01-stop-maintenance [production]
13:51 <rzl@cumin1001> END (PASS) - Cookbook sre.switchdc.mediawiki.00-warmup-caches (exit_code=0) [production]
13:50 <rzl@cumin1001> START - Cookbook sre.switchdc.mediawiki.00-warmup-caches [production]
13:45 <rzl@cumin1001> END (PASS) - Cookbook sre.switchdc.mediawiki.00-warmup-caches (exit_code=0) [production]
13:44 <rzl@cumin1001> START - Cookbook sre.switchdc.mediawiki.00-warmup-caches [production]
13:40 <rzl@cumin1001> END (PASS) - Cookbook sre.switchdc.mediawiki.00-warmup-caches (exit_code=0) [production]
13:39 <rzl@cumin1001> START - Cookbook sre.switchdc.mediawiki.00-warmup-caches [production]
13:37 <rzl@cumin1001> END (PASS) - Cookbook sre.switchdc.mediawiki.00-reduce-ttl (exit_code=0) [production]