|
2020-03-30
ยง
|
| 15:25 |
<jeh> |
add icinga 2h downtime and soft reset iDRAC on labstore1005.mgmt.eqiad.wmnet T247965 |
[production] |
| 14:58 |
<vgutierrez@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 14:57 |
<vgutierrez@cumin1001> |
END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) |
[production] |
| 14:57 |
<vgutierrez@cumin1001> |
START - Cookbook sre.hosts.decommission |
[production] |
| 14:55 |
<vgutierrez@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 14:53 |
<vgutierrez> |
depool & decommission cp2008 - T248864 |
[production] |
| 14:23 |
<vgutierrez> |
pool cp2032 - T248816 |
[production] |
| 14:17 |
<vgutierrez@cumin1001> |
END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) |
[production] |
| 14:17 |
<vgutierrez@cumin1001> |
START - Cookbook sre.hosts.decommission |
[production] |
| 14:01 |
<vgutierrez> |
depool & decommission cp2006 - T248856 |
[production] |
| 13:57 |
<vgutierrez@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 13:45 |
<vgutierrez> |
pool cp2031 - T248816 |
[production] |
| 13:09 |
<vgutierrez@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 13:07 |
<dzahn@cumin1001> |
END (FAIL) - Cookbook sre.hosts.decommission (exit_code=99) |
[production] |
| 13:07 |
<dzahn@cumin1001> |
START - Cookbook sre.hosts.decommission |
[production] |
| 13:06 |
<vgutierrez@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 12:56 |
<vgutierrez@cumin1001> |
END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) |
[production] |
| 12:56 |
<vgutierrez@cumin1001> |
START - Cookbook sre.hosts.decommission |
[production] |
| 12:53 |
<vgutierrez> |
depool & decommission cp2005 - T248848 |
[production] |
| 12:26 |
<cdanis> |
cdanis@re0.cr2-codfw# set chassis fpc 5 inline-services flex-flow-sizing cdanis@re0.cr2-codfw# commit comment "flex-flow-sizing T248394" |
[production] |
| 12:24 |
<vgutierrez@cumin1001> |
END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) |
[production] |
| 12:23 |
<vgutierrez@cumin1001> |
START - Cookbook sre.hosts.decommission |
[production] |
| 12:21 |
<vgutierrez> |
depool & decommission cp2004 - T248824 |
[production] |
| 12:03 |
<XioNoX> |
delete unused ROA for ARIN v6 prefixes - T235886 |
[production] |
| 11:59 |
<XioNoX> |
delete unused ROAs for RIPE prefixes - T235886 |
[production] |
| 11:42 |
<mutante> |
miscweb2002 - race condition with apache2 mpm and php7.3 module met - a2dismond mpm_event ; systemctl restart apache2 ; puppet agent -tv (also see T196968, https://gerrit.wikimedia.org/r/c/operations/puppet/+/451206) T247887 |
[production] |
| 11:37 |
<mutante> |
miscweb2002 - installed OS, added to puppet, added role and ... sed -i 's/tin.eqiad/deployment.eqiad/g' /srv/deployment/iegreview/iegreview-cache/.config (T247648) |
[production] |
| 11:30 |
<marostegui> |
Deploy schema change on dbstore1004:3314 |
[production] |
| 11:22 |
<XioNoX> |
delete ARIN allocations from RIPE's IRR - T235886 |
[production] |
| 11:11 |
<Urbanecm> |
EU SWAT done |
[production] |
| 11:10 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: ac7e625: Add collections.nmnh.si.edu to $wgCopyUploadsDomains (T248659; take II) (duration: 00m 58s) |
[production] |
| 11:09 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: ac7e625: Add collections.nmnh.si.edu to $wgCopyUploadsDomains (T248659) (duration: 00m 58s) |
[production] |
| 11:08 |
<vgutierrez> |
pool cp2030 - T248816 |
[production] |
| 11:07 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: c8c06f9: Add 3 additional namespaces and assoicated talk pages to trwiktionary (T248734; take II) (duration: 00m 59s) |
[production] |
| 11:06 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: c8c06f9: Add 3 additional namespaces and assoicated talk pages to trwiktionary (T248734) (duration: 00m 59s) |
[production] |
| 10:43 |
<dzahn@cumin1001> |
END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) |
[production] |
| 10:34 |
<dzahn@cumin1001> |
START - Cookbook sre.ganeti.makevm |
[production] |
| 10:33 |
<dzahn@cumin1001> |
END (FAIL) - Cookbook sre.ganeti.makevm (exit_code=99) |
[production] |
| 10:33 |
<dzahn@cumin1001> |
START - Cookbook sre.ganeti.makevm |
[production] |
| 09:59 |
<hoo> |
Temporary modified dumpsgen's crontab on snapshot1008 so that the Wikidata JSON dumps start at 9:59 UTC today (T248612) |
[production] |
| 09:56 |
<hoo@deploy1001> |
Synchronized php-1.35.0-wmf.25/extensions/Wikibase/repo/maintenance/DumpEntities.php: DumpEntities: Fix DB group default override (T248612) (duration: 01m 02s) |
[production] |
| 09:19 |
<vgutierrez@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 09:15 |
<vgutierrez@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 08:30 |
<vgutierrez> |
pool cp2029 - T248816 |
[production] |
| 08:12 |
<vgutierrez@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 08:12 |
<vgutierrez@cumin1001> |
END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) |
[production] |
| 08:12 |
<vgutierrez@cumin1001> |
START - Cookbook sre.hosts.decommission |
[production] |
| 08:10 |
<vgutierrez@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 07:53 |
<vgutierrez> |
depool & decommission cp2002 - T248818 |
[production] |
| 07:48 |
<marostegui> |
Run cloudcontrol1003:~# wmcs-wikireplica-dns to promote dbproxy1018 to wikireplicas active proxy T231520 |
[production] |