|
2020-05-15
§
|
| 14:04 |
<pt1979@cumin2001> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) |
[production] |
| 14:01 |
<pt1979@cumin2001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 13:47 |
<ema> |
cp2029, cp3050: varnish-fe-restart to clear 'child restarted' alerts |
[production] |
| 13:47 |
<vgutierrez> |
downgrade ats to version 8.0.7-1wm7 on cp4032 |
[production] |
| 13:42 |
<vgutierrez> |
upgrade ats to version 8.0.7-1wm8 on cp4032 |
[production] |
| 13:37 |
<mutante> |
rsyncing gerrit git data from gerrit1001 to gerrit1002 (T200739) |
[production] |
| 13:13 |
<cdanis> |
increase samplicator recvbuf on netflow3001 & restart samplicator |
[production] |
| 13:01 |
<cdanis> |
increasing sysctl net.ipv4.udp_mem on netflow3001 |
[production] |
| 09:57 |
<vgutierrez> |
upload trafficserver 8.0.7-1wm7 to apt.wm.o (buster) |
[production] |
| 09:21 |
<ema> |
cp2029: attempt forced discard of stuck VCL T236754 |
[production] |
| 09:09 |
<elukey> |
restart druid brokers on druid100[4-6] - locked up due to datasources dropped - T226035 |
[production] |
| 08:51 |
<ema> |
cp2029: try out varnish 5.1.3-1wm15 T236754 |
[production] |
| 07:36 |
<XioNoX> |
bumps prefix limit for AS16735 in eqiad |
[production] |
| 05:35 |
<jynus> |
stop replication on pc2009, pc2010 for benchmarking T252761 |
[production] |
| 04:53 |
<volker-e@deploy1001> |
Finished deploy [design/style-guide@dc956a3]: Deploy design/style-guide: (duration: 00m 10s) |
[production] |
| 04:52 |
<volker-e@deploy1001> |
Started deploy [design/style-guide@dc956a3]: Deploy design/style-guide: |
[production] |
| 04:42 |
<vgutierrez> |
repool cp5006 |
[production] |
| 04:28 |
<vgutierrez> |
depool and reboot cp5006 |
[production] |
|
2020-05-14
§
|
| 23:24 |
<catrope@deploy1001> |
Synchronized static/images/project-logos/: Revert temporary 20k logo for vecwiki (T252770) (duration: 01m 06s) |
[production] |
| 23:23 |
<RoanKattouw> |
Ran namespaceDupes.php for T252343 |
[production] |
| 23:20 |
<catrope@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: Create Gapura (Portal) namespace on jvwiki (T252343) (duration: 01m 06s) |
[production] |
| 23:09 |
<catrope@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: Add *.ub.uni-heidelberg.de and hq.eso.org to $wgCopyUploadDomains (T252600, T252726) (duration: 01m 07s) |
[production] |
| 21:43 |
<ryankemper> |
depooled wdqs2006 while lag recovers |
[production] |
| 21:42 |
<ryankemper@cumin1001> |
END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) |
[production] |
| 21:08 |
<ryankemper@cumin2001> |
END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) |
[production] |
| 20:16 |
<volans> |
moved codereview.tar.gz and with_r.tar.gz from miscweb1002 to cumin1001 to free space |
[production] |
| 20:15 |
<hashar@deploy1001> |
Synchronized php-1.35.0-wmf.32/skins/Vector/includes/VectorTemplate.php: Allow plain text labels in side bar - T252727 (duration: 01m 06s) |
[production] |
| 19:51 |
<ryankemper@cumin2001> |
START - Cookbook sre.wdqs.data-transfer |
[production] |
| 19:50 |
<ryankemper@cumin1001> |
START - Cookbook sre.wdqs.data-transfer |
[production] |
| 19:49 |
<ryankemper> |
Depooled wqds1006 in preparation for impending wdqs data xfer |
[production] |
| 18:36 |
<Urbanecm> |
Morning SWAT done |
[production] |
| 18:35 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: 15adbbc: [thwikisource] Set ProofReadPage separator to an empty string (T252610) (duration: 01m 06s) |
[production] |
| 18:26 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: 4b8399c: Undeploy graphoid from mediawikiwiki (T242855) (duration: 01m 05s) |
[production] |
| 18:16 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: f03a45c: Adding import to test wikis from mediawikiwiki (T242855) (duration: 01m 07s) |
[production] |
| 17:02 |
<XioNoX> |
asw2-d-eqiad> request virtual-chassis vc-port delete pic-slot 1 port 1 member 1 - T252797 |
[production] |
| 16:55 |
<XioNoX> |
asw2-d-eqiad> request virtual-chassis vc-port delete pic-slot 1 port 3 member 1 - T252797 |
[production] |
| 16:51 |
<XioNoX> |
asw2-d-eqiad> request virtual-chassis vc-port set pic-slot 0 port 48 member 2 - T252797 |
[production] |
| 16:49 |
<XioNoX> |
request virtual-chassis vc-port set pic-slot 1 port 2 member 1 - T252797 |
[production] |
| 16:42 |
<XioNoX> |
request virtual-chassis vc-port delete pic-slot 1 port 2 member 1 - T252797 |
[production] |
| 16:36 |
<XioNoX> |
asw2-d-eqiad> request virtual-chassis vc-port delete pic-slot 0 port 48 member 2 - T252797 |
[production] |
| 15:59 |
<hnowlan@deploy1001> |
helmfile [CODFW] Ran 'sync' command on namespace 'changeprop' for release 'production' . |
[production] |
| 15:57 |
<hnowlan@deploy1001> |
helmfile [EQIAD] Ran 'sync' command on namespace 'changeprop' for release 'production' . |
[production] |
| 15:56 |
<hnowlan@deploy1001> |
helmfile [STAGING] Ran 'sync' command on namespace 'changeprop' for release 'staging' . |
[production] |
| 15:25 |
<XioNoX> |
disable asw2-d1-eqiad:et-1/1/0 - T251663 |
[production] |
| 14:39 |
<mutante> |
kuai kuai is https://twitter.com/Arlieth/status/1257714333133357056 | https://en.wikipedia.org/wiki/Kuai_Kuai_culture |
[production] |
| 13:31 |
<_joe_> |
updating purged to 0.11 in eqiad,eqsin,esams |
[production] |
| 12:47 |
<vgutierrez> |
rolling upgrade ats to version 8.0.7-1wm7 |
[production] |
| 12:46 |
<elukey@cumin1001> |
END (PASS) - Cookbook sre.aqs.roll-restart (exit_code=0) |
[production] |
| 12:43 |
<elukey@cumin1001> |
START - Cookbook sre.aqs.roll-restart |
[production] |
| 12:22 |
<kormat> |
reverted iosched on pc1010 to `mq-deadline` T252761 |
[production] |