801-850 of 10000 results (33ms)
2021-07-15 ยง
16:46 <hnowlan@puppetmaster1001> conftool action : set/pooled=yes; selector: name=maps20(0[1-6]|10).codfw.wmnet [production]
16:43 <otto@deploy1002> Started deploy [analytics/refinery@7a673c9]: Deploy refinery-source 0.1.15 with fixes for Refine jobs [production]
16:40 <ejegg> updated payments-wiki from d9892207c1 to 844b59ee42 [production]
16:39 <hnowlan@puppetmaster1001> conftool action : set/pooled=no; selector: name=maps20(0[1-6]|10).codfw.wmnet [production]
16:39 <hnowlan@puppetmaster1001> conftool action : set/pooled=yes; selector: name=maps200[7-9].codfw.wmnet [production]
16:27 <ejegg> updated fundraising CiviCRM from e0d53c92b5 to 20e9ef6bbb [production]
16:24 <ejegg> updated payments-wiki from 0e7800027a to 844b59ee42 [production]
16:19 <hnowlan@puppetmaster1001> conftool action : set/pooled=no; selector: name=maps2007.codfw.wmnet [production]
16:00 <kormat@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:30:00 on es1029.eqiad.wmnet with reason: Rebooting for T273281 [production]
16:00 <kormat@cumin1001> START - Cookbook sre.hosts.downtime for 1:30:00 on es1029.eqiad.wmnet with reason: Rebooting for T273281 [production]
15:51 <kormat@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:30:00 on es1028.eqiad.wmnet with reason: Rebooting for T273281 [production]
15:51 <kormat@cumin1001> START - Cookbook sre.hosts.downtime for 1:30:00 on es1028.eqiad.wmnet with reason: Rebooting for T273281 [production]
15:19 <kormat@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:30:00 on es1027.eqiad.wmnet with reason: Rebooting for T273281 [production]
15:19 <kormat@cumin1001> START - Cookbook sre.hosts.downtime for 1:30:00 on es1027.eqiad.wmnet with reason: Rebooting for T273281 [production]
15:16 <ladsgroup@deploy1002> Synchronized wmf-config/flaggedrevs.php: Config: [[gerrit:704773|flaggedrevs: Allow admins of idwiki to change stablesettings (T268317)]], try II (duration: 01m 05s) [production]
15:03 <Amir1> temporary becoming admin on idwiki to debug T268317 [production]
15:02 <moritzm> installing nginx security updates on ms-fe* [production]
14:47 <kormat@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:30:00 on es1026.eqiad.wmnet with reason: Rebooting for T273281 [production]
14:47 <kormat@cumin1001> START - Cookbook sre.hosts.downtime for 1:30:00 on es1026.eqiad.wmnet with reason: Rebooting for T273281 [production]
14:40 <hnowlan@puppetmaster1001> conftool action : set/pooled=no; selector: name=maps200[7-9].codfw.wmnet [production]
14:40 <hnowlan@puppetmaster1001> conftool action : set/pooled=yes; selector: name=maps20(0[1-6]|10).codfw.wmnet [production]
14:33 <hnowlan@puppetmaster1001> conftool action : set/pooled=no; selector: name=maps20(0[1-6]|10).codfw.wmnet [production]
14:33 <hnowlan@puppetmaster1001> conftool action : set/pooled=yes; selector: name=maps200[7-9].codfw.wmnet [production]
14:11 <kormat@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:30:00 on es1024.eqiad.wmnet with reason: Rebooting for T273281 [production]
14:11 <kormat@cumin1001> START - Cookbook sre.hosts.downtime for 1:30:00 on es1024.eqiad.wmnet with reason: Rebooting for T273281 [production]
14:11 <kormat@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:30:00 on es2023.codfw.wmnet,es[1023-1025].eqiad.wmnet with reason: Rebooting es1024 (es5 eqiad primary) for kernel upgrade T273281 [production]
14:11 <kormat@cumin1001> START - Cookbook sre.hosts.downtime for 1:30:00 on es2023.codfw.wmnet,es[1023-1025].eqiad.wmnet with reason: Rebooting es1024 (es5 eqiad primary) for kernel upgrade T273281 [production]
14:10 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts mw1301.eqiad.wmnet [production]
13:57 <hnowlan@puppetmaster1001> conftool action : set/weight=10; selector: name=maps2009.codfw.wmnet [production]
13:47 <hnowlan@puppetmaster1001> conftool action : set/pooled=no; selector: name=maps200[7-9].codfw.wmnet [production]
13:47 <hnowlan@puppetmaster1001> conftool action : set/pooled=yes; selector: name=maps20(0[1-6]|10).codfw.wmnet [production]
13:46 <kormat@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:30:00 on es1021.eqiad.wmnet with reason: Rebooting for T273281 [production]
13:46 <kormat@cumin1001> START - Cookbook sre.hosts.downtime for 1:30:00 on es1021.eqiad.wmnet with reason: Rebooting for T273281 [production]
13:46 <kormat@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:30:00 on es2021.codfw.wmnet,es[1020-1022].eqiad.wmnet with reason: Rebooting es1021 (es4 eqiad primary) for kernel upgrade T273281 [production]
13:46 <kormat@cumin1001> START - Cookbook sre.hosts.downtime for 1:30:00 on es2021.codfw.wmnet,es[1020-1022].eqiad.wmnet with reason: Rebooting es1021 (es4 eqiad primary) for kernel upgrade T273281 [production]
13:45 <hnowlan@puppetmaster1001> conftool action : set/pooled=no; selector: name=maps20(0[1-6]|10).codfw.wmnet [production]
13:45 <hnowlan@puppetmaster1001> conftool action : set/pooled=yes; selector: name=maps200[7-9].codfw.wmnet [production]
13:38 <dzahn@cumin1001> START - Cookbook sre.hosts.decommission for hosts mw1301.eqiad.wmnet [production]
13:34 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts mw1300.eqiad.wmnet [production]
13:33 <dzahn@cumin1001> conftool action : set/pooled=inactive; selector: name=mw130[0-1].eqiad.wmnet [production]
13:22 <mutante> mw1300, mw1301 - jobrunners going out of service, decom [production]
13:21 <dzahn@cumin1001> START - Cookbook sre.hosts.decommission for hosts mw1300.eqiad.wmnet [production]
13:20 <jelto@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
13:17 <jelto@cumin1001> START - Cookbook sre.dns.netbox [production]
13:12 <dcausse@deploy1002> helmfile [staging] Ran 'sync' command on namespace 'rdf-streaming-updater' for release 'main' . [production]
13:10 <dzahn@cumin1001> conftool action : set/pooled=no; selector: name=mw130[0-1].eqiad.wmnet [production]
13:06 <dzahn@cumin1001> conftool action : set/pooled=yes; selector: name=mw1413.eqiad.wmnet [production]
13:05 <mutante> mw1413 - pooling, was depooled but for unknown reason, dont see it in SAL, looks ok, scap pulled [production]
13:03 <dzahn@cumin1001> conftool action : set/weight=30; selector: name=mw1422.eqiad.wmnet [production]
13:03 <kormat@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:30:00 on db1165.eqiad.wmnet with reason: Rebooting for T273281 [production]