451-500 of 10000 results (16ms)
2020-08-12 §
12:52 <oblivian@cumin1001> START - Cookbook sre.hosts.reboot-single [production]
12:50 <oblivian@cumin1001> START - Cookbook sre.hosts.reboot-single [production]
12:33 <oblivian@cumin1001> END (FAIL) - Cookbook sre.hosts.reboot-single (exit_code=1) [production]
12:27 <oblivian@cumin1001> START - Cookbook sre.hosts.reboot-single [production]
12:20 <oblivian@cumin1001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) [production]
12:20 <oblivian@cumin1001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) [production]
12:16 <oblivian@cumin1001> START - Cookbook sre.hosts.reboot-single [production]
12:15 <oblivian@cumin1001> START - Cookbook sre.hosts.reboot-single [production]
12:15 <oblivian@cumin1001> END (FAIL) - Cookbook sre.hosts.reboot-single (exit_code=99) [production]
12:15 <oblivian@cumin1001> START - Cookbook sre.hosts.reboot-single [production]
11:51 <ema> pool mw1363 after reboot [production]
11:49 <jynus> creating artificial low replication lag on db2130 to test icinga alerts T253120 [production]
11:41 <ema@cumin1001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) [production]
11:37 <ema@cumin1001> START - Cookbook sre.hosts.reboot-single [production]
11:30 <jayme@cumin1001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) [production]
11:28 <oblivian@cumin1001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) [production]
11:25 <jayme@cumin1001> START - Cookbook sre.hosts.reboot-single [production]
11:24 <oblivian@cumin1001> START - Cookbook sre.hosts.reboot-single [production]
11:21 <oblivian@cumin1001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) [production]
11:17 <oblivian@cumin1001> START - Cookbook sre.hosts.reboot-single [production]
11:17 <jayme@cumin1001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) [production]
11:13 <oblivian@cumin1001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) [production]
11:10 <jayme@cumin1001> START - Cookbook sre.hosts.reboot-single [production]
11:08 <oblivian@cumin1001> START - Cookbook sre.hosts.reboot-single [production]
11:07 <oblivian@cumin1001> END (FAIL) - Cookbook sre.hosts.reboot-single (exit_code=99) [production]
11:07 <oblivian@cumin1001> START - Cookbook sre.hosts.reboot-single [production]
11:00 <oblivian@cumin1001> END (FAIL) - Cookbook sre.hosts.reboot-single (exit_code=99) [production]
11:00 <oblivian@cumin1001> START - Cookbook sre.hosts.reboot-single [production]
10:55 <_joe_> rebooting mw1361 [production]
10:51 <jayme> rebooting mw1356 [production]
10:49 <_joe_> rebooting mw1378 [production]
09:45 <_joe_> repooling mw1377 [production]
09:40 <_joe_> rebooting mw1377 [production]
09:22 <_joe_> depool mw1357 tool [production]
09:14 <_joe_> depooling mw1377 for inspection [production]
09:12 <marostegui@cumin1001> dbctl commit (dc=all): 'Fully repool db1110', diff saved to https://phabricator.wikimedia.org/P12220 and previous config saved to /var/cache/conftool/dbconfig/20200812-091211-marostegui.json [production]
09:08 <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1110', diff saved to https://phabricator.wikimedia.org/P12219 and previous config saved to /var/cache/conftool/dbconfig/20200812-090831-marostegui.json [production]
08:50 <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1110', diff saved to https://phabricator.wikimedia.org/P12218 and previous config saved to /var/cache/conftool/dbconfig/20200812-085021-marostegui.json [production]
08:35 <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1110', diff saved to https://phabricator.wikimedia.org/P12217 and previous config saved to /var/cache/conftool/dbconfig/20200812-083548-marostegui.json [production]
07:49 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
07:46 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime [production]
07:31 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1110 for reimage', diff saved to https://phabricator.wikimedia.org/P12215 and previous config saved to /var/cache/conftool/dbconfig/20200812-073130-marostegui.json [production]
04:51 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1126 for MCR change', diff saved to https://phabricator.wikimedia.org/P12214 and previous config saved to /var/cache/conftool/dbconfig/20200812-045157-marostegui.json [production]
2020-08-11 §
23:41 <Urbanecm> Evening B&C window completed [production]
23:39 <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: 0f238f71c95c7bd7534c28abfac759fbb47f674f: Update wgMFRemovableClasses (T231160) (duration: 01m 03s) [production]
23:36 <urbanecm@deploy1001> Synchronized php-1.36.0-wmf.3/extensions/MobileFrontend/extension.json: c22d65ff9b2439f484ab8ccffed87b00e78c3ad2: Hide vertical nav-boxes on mobile domain (T231160) (duration: 01m 03s) [production]
23:34 <urbanecm@deploy1001> Synchronized php-1.36.0-wmf.4/extensions/MobileFrontend/extension.json: 81d54b0ec82d0b78f723f9400031e918a4a143aa: Hide vertical nav-boxes on mobile domain (T231160) (duration: 01m 05s) [production]
23:07 <urbanecm@deploy1001> Synchronized wmf-config/CommonSettings.php: 28faa279dacf6a4d6f0a663844e913738c2fa142: Switching to updated license definition (duration: 01m 04s) [production]
21:52 <krinkle@deploy1001> Synchronized php-1.36.0-wmf.3/includes/skins/SkinMustache.php: Ibe1f07346, T259872, T259858 (duration: 01m 04s) [production]
19:40 <otto@deploy1001> helmfile [staging] Ran 'sync' command on namespace 'eventgate-main' for release 'production' . [production]