|
2020-09-14
ยง
|
| 16:48 |
<dzahn@cumin1001> |
conftool action : set/pooled=no; selector: dc=codfw,cluster=parsoid,name=parse2001.codfw.wmnet |
[production] |
| 16:36 |
<mutante> |
pooled the first of the new parsoid servers - parse2001 (T247441) |
[production] |
| 16:33 |
<dzahn@cumin1001> |
conftool action : set/pooled=yes; selector: dc=codfw,cluster=parsoid,name=parse2001.codfw.wmnet |
[production] |
| 16:07 |
<dzahn@cumin1001> |
conftool action : set/pooled=no; selector: dc=codfw,cluster=parsoid,name=parse20[1-2][0-9].codfw.wmnet |
[production] |
| 16:04 |
<elukey> |
completed the rollout of restrictive kafka ferm rules on the Kafka jumbo cluster |
[production] |
| 16:03 |
<dzahn@cumin1001> |
conftool action : set/pooled=no; selector: dc=codfw,cluster=parsoid,name=parse200[0-9].codfw.wmnet |
[production] |
| 16:01 |
<dzahn@cumin1001> |
conftool action : set/weight=10; selector: dc=codfw,cluster=parsoid,name=parse20[0-2][0-9].codfw.wmnet |
[production] |
| 15:59 |
<dzahn@cumin1001> |
conftool action : set/pooled=no; selector: dc=codfw,cluster=parsoid,name=parse2001.codfw.wmnet |
[production] |
| 15:58 |
<dzahn@cumin1001> |
conftool action : set/weight=10; selector: dc=codfw,cluster=parsoid,name=parse20[1-2][0-9].codfw.wmnet |
[production] |
| 15:54 |
<moritzm> |
restarting apache on webperf* to pick up GNU TLS security update |
[production] |
| 15:45 |
<moritzm> |
restarting apache/FPM on mw2271/m2272 (codfw canaries) to pick up GNU TLS update |
[production] |
| 15:35 |
<moritzm> |
installing gnutls28 security updates on stretch |
[production] |
| 15:23 |
<elukey> |
enable stricter ferm rules on kafka-jumbo1007 and kafka-jumbo1005 |
[production] |
| 15:17 |
<cicalese@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:626229|Allow public access to API Portal main page for private launch]] (duration: 00m 57s) |
[production] |
| 15:17 |
<volans@cumin1001> |
END (PASS) - Cookbook sre.dns.netbox (exit_code=0) |
[production] |
| 15:11 |
<volans@cumin1001> |
START - Cookbook sre.dns.netbox |
[production] |
| 15:11 |
<cmjohnson1> |
completed pdu swap in eqiad racks d5/d6 |
[production] |
| 14:55 |
<elukey> |
ferm rules added to kafka-jumbo1009, 1006 and 1008 up to now |
[production] |
| 14:24 |
<milimetric@deploy1001> |
helmfile [codfw] Ran 'sync' command on namespace 'eventstreams' for release 'canary' . |
[production] |
| 14:24 |
<milimetric@deploy1001> |
helmfile [codfw] Ran 'sync' command on namespace 'eventstreams' for release 'production' . |
[production] |
| 14:16 |
<volans@cumin1001> |
END (FAIL) - Cookbook sre.dns.netbox (exit_code=99) |
[production] |
| 14:14 |
<milimetric@deploy1001> |
helmfile [eqiad] Ran 'sync' command on namespace 'eventstreams' for release 'production' . |
[production] |
| 14:14 |
<milimetric@deploy1001> |
helmfile [eqiad] Ran 'sync' command on namespace 'eventstreams' for release 'canary' . |
[production] |
| 14:11 |
<volans@cumin1001> |
START - Cookbook sre.dns.netbox |
[production] |
| 14:09 |
<milimetric@deploy1001> |
helmfile [staging] Ran 'sync' command on namespace 'eventstreams' for release 'canary' . |
[production] |
| 14:09 |
<milimetric@deploy1001> |
helmfile [staging] Ran 'sync' command on namespace 'eventstreams' for release 'production' . |
[production] |
| 13:55 |
<volans@cumin1001> |
END (PASS) - Cookbook sre.dns.netbox (exit_code=0) |
[production] |
| 13:50 |
<volans@cumin1001> |
START - Cookbook sre.dns.netbox |
[production] |
| 13:42 |
<moritzm> |
installing dbus security updates on stretch |
[production] |
| 13:42 |
<volans@cumin1001> |
END (FAIL) - Cookbook sre.dns.netbox (exit_code=99) |
[production] |
| 13:32 |
<moritzm> |
installing websockify stretch updates |
[production] |
| 13:10 |
<volans@cumin1001> |
START - Cookbook sre.dns.netbox |
[production] |
| 12:51 |
<cmjohnson1> |
correction it's replacing the pdu's in racks d5 and d6 |
[production] |
| 12:50 |
<Amir1> |
ladsgroup@mwmaint2001:~$ mwscript extensions/Wikibase/repo/maintenance/changePropertyDataType.php --wiki=wikidatawiki --property-id P1438 --new-data-type external-id (T262198) |
[production] |
| 12:49 |
<cmjohnson1> |
replacing pdu's in racks d4 and d5 eqiad |
[production] |
| 12:32 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) |
[production] |
| 12:32 |
<ayounsi@cumin1001> |
END (FAIL) - Cookbook sre.pdus.rotate-snmp (exit_code=1) |
[production] |
| 12:30 |
<ayounsi@cumin1001> |
START - Cookbook sre.pdus.rotate-snmp |
[production] |
| 12:30 |
<XioNoX> |
rotate SNMP community on all the PDUs - T246890 |
[production] |
| 12:24 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.reboot-single |
[production] |
| 12:24 |
<moritzm> |
rebooting sodium for kernel update |
[production] |
| 12:09 |
<volans@cumin1001> |
END (FAIL) - Cookbook sre.hosts.decommission (exit_code=1) |
[production] |
| 12:08 |
<volans@cumin1001> |
START - Cookbook sre.hosts.decommission |
[production] |
| 12:06 |
<akosiaris> |
T187984 migration script on otrs1001 now in step 31/44 |
[production] |
| 12:03 |
<volans@cumin1001> |
END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) |
[production] |
| 11:53 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: fea8861db550746bfef496df2ef522dffc580a7d: Follow-up 0ee0d8f: [frwiktionary] Create `conj` alias (T262298) (duration: 00m 56s) |
[production] |
| 11:50 |
<volans@cumin1001> |
START - Cookbook sre.ganeti.makevm |
[production] |
| 11:48 |
<volans@cumin1001> |
END (FAIL) - Cookbook sre.ganeti.makevm (exit_code=99) |
[production] |
| 11:48 |
<volans@cumin1001> |
START - Cookbook sre.ganeti.makevm |
[production] |
| 11:46 |
<volans@cumin1001> |
END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) |
[production] |