3601-3650 of 10000 results (37ms)
2021-01-22 §
00:48 <robh@cumin1001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on db1169.eqiad.wmnet with reason: REIMAGE [production]
00:48 <robh@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1162.eqiad.wmnet with reason: REIMAGE [production]
00:46 <robh@cumin1001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on db1164.eqiad.wmnet with reason: REIMAGE [production]
00:46 <robh@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1166.eqiad.wmnet with reason: REIMAGE [production]
00:44 <robh@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on db1174.eqiad.wmnet with reason: REIMAGE [production]
00:44 <robh@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on db1156.eqiad.wmnet with reason: REIMAGE [production]
00:44 <robh@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on db1161.eqiad.wmnet with reason: REIMAGE [production]
00:44 <robh@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on db1157.eqiad.wmnet with reason: REIMAGE [production]
00:44 <robh@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on db1167.eqiad.wmnet with reason: REIMAGE [production]
00:44 <robh@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on db1163.eqiad.wmnet with reason: REIMAGE [production]
00:44 <robh@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on db1158.eqiad.wmnet with reason: REIMAGE [production]
00:44 <robh@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on db1165.eqiad.wmnet with reason: REIMAGE [production]
00:44 <robh@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on db1160.eqiad.wmnet with reason: REIMAGE [production]
00:44 <robh@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on db1170.eqiad.wmnet with reason: REIMAGE [production]
00:44 <robh@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on db1169.eqiad.wmnet with reason: REIMAGE [production]
00:44 <robh@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on db1168.eqiad.wmnet with reason: REIMAGE [production]
00:44 <robh@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on db1164.eqiad.wmnet with reason: REIMAGE [production]
00:44 <robh@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on db1162.eqiad.wmnet with reason: REIMAGE [production]
00:44 <robh@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on db1166.eqiad.wmnet with reason: REIMAGE [production]
00:41 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mw2376.codfw.wmnet with reason: REIMAGE [production]
00:39 <dzahn@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on mw2376.codfw.wmnet with reason: REIMAGE [production]
00:37 <dzahn@cumin1001> conftool action : set/pooled=yes; selector: name=mw2372.codfw.wmnet [production]
00:35 <dzahn@cumin1001> conftool action : set/pooled=yes; selector: name=mw2370.codfw.wmnet [production]
00:31 <brennen@deploy1001> rebuilt and synchronized wikiversions files: all wikis to 1.36.0-wmf.27 [production]
00:27 <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: d4f5d6f09977962be1c49471432125a92357ede6: Temporarily amend ukwiki AF configuration (T272330) (duration: 01m 03s) [production]
00:20 <brennen@deploy1001> Synchronized php-1.36.0-wmf.27/extensions/MobileFrontend: Backport: [[gerrit:657702|Fix toggling storage cleanup (T272638)]] (duration: 01m 07s) [production]
00:18 <dzahn@cumin1001> conftool action : set/pooled=no; selector: name=mw2372.codfw.wmnet [production]
00:10 <dzahn@cumin1001> conftool action : set/pooled=no; selector: name=mw2370.codfw.wmnet [production]
00:03 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on mw2370.codfw.wmnet with reason: new install on buster [production]
00:03 <dzahn@cumin1001> START - Cookbook sre.hosts.downtime for 4:00:00 on mw2370.codfw.wmnet with reason: new install on buster [production]
2021-01-21 §
23:57 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mw2372.codfw.wmnet with reason: REIMAGE [production]
23:55 <dzahn@cumin1001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on mw2370.codfw.wmnet with reason: REIMAGE [production]
23:55 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mw2374.codfw.wmnet with reason: REIMAGE [production]
23:54 <dzahn@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on mw2370.codfw.wmnet with reason: REIMAGE [production]
23:53 <dzahn@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on mw2372.codfw.wmnet with reason: REIMAGE [production]
23:53 <dzahn@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on mw2374.codfw.wmnet with reason: REIMAGE [production]
22:10 <brennen> 1.36.0-wmf.27 train status: for avoidance of doubt, no deploys until further notice - sorting out T272638 [production]
21:27 <brennen@deploy1001> rebuilt and synchronized wikiversions files: Revert group2 wikis to 1.36.0-wmf.26 [production]
20:06 <brennen@deploy1001> rebuilt and synchronized wikiversions files: all wikis to 1.36.0-wmf.27 [production]
20:04 <razzi@cumin1001> END (PASS) - Cookbook sre.druid.reboot-workers (exit_code=0) for Druid public cluster: Reboot Druid nodes - razzi@cumin1001 [production]
19:51 <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: ac99da75f9507e19472ab3020be638262857ec07: Migrate WebUIActionsTracking schemas to Event Platform on testwiki (T267347; T271164) (duration: 01m 03s) [production]
19:41 <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: 4bb9e5d13be702516368774732a9e1711bec42e5: Enables the Wikisource extension on oldwikisource (T272163) (duration: 01m 04s) [production]
19:40 <urbanecm@deploy1001> Synchronized php-1.36.0-wmf.27/extensions/EventLogging/: ee830a5ec2051fa970084e89b477a44c384e309c: f7152a74e00404fc561c44d1c2e37d7f882e2f52: EventLogging backport, see commits for details (T253121) (duration: 01m 05s) [production]
19:39 <dzahn@cumin1001> conftool action : set/pooled=yes; selector: name=mw2226.codfw.wmnet [production]
19:38 <dzahn@cumin1001> conftool action : set/pooled=yes; selector: name=mw2375.codfw.wmnet [production]
19:38 <dzahn@cumin1001> conftool action : set/pooled=yes; selector: name=mw2373.codfw.wmnet [production]
19:38 <dzahn@cumin1001> conftool action : set/pooled=yes; selector: name=mw2371.codfw.wmnet [production]
19:38 <dzahn@cumin1001> conftool action : set/pooled=no; selector: name=mw2226.codfw.wmnet [production]
19:28 <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: 62c9c35a76e2d065922f8c9f5a58672240dea7de: Migrate SuggestedTagsAction to Event Platform on all wikis (T267351) (duration: 01m 03s) [production]
19:21 <urbanecm@deploy1001> Synchronized wmf-config/CommonSettings.php: 0b46c9f1f75fc773f57bfa70521c9eaf20410b9e: [no-op] Add notes about load order of Wikisource and Collection extensions (T255790) (duration: 01m 11s) [production]