|
2021-01-25
§
|
| 12:25 |
<joal> |
Copy /wmf/data/archive to backup cluster (32Tb) - T272846 |
[analytics] |
| 12:23 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: 177339d96616b5941dbeb2c90ca6aa0be90e3b5a: Defining wgSitename for trwikivoyage (T272779) (duration: 01m 00s) |
[production] |
| 12:19 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: 89d072378e16b0410d963deca2fd766c1406b5b6: Enable SandboxLink on Turkish Wikivoyage (T272780) (duration: 01m 05s) |
[production] |
| 12:15 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: 75aa32fd5aee1feebe8a97360068da55cbcf06d8: frwiki: Change back to normal logo (T272700) (duration: 01m 07s) |
[production] |
| 12:07 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: 693eaec20a24620c2a709c8bac707c0d7af3436b: Add bidgee.id.au to the wgCopyUploadsDomains allowlist of Wikimedia Commons (T272202) (duration: 01m 01s) |
[production] |
| 11:40 |
<jdrewniak@deploy1001> |
Synchronized portals: Wikimedia Portals Update: [[gerrit:658242| Bumping portals to master (T128546)]] (duration: 00m 55s) |
[production] |
| 11:39 |
<jdrewniak@deploy1001> |
Synchronized portals/wikipedia.org/assets: Wikimedia Portals Update: [[gerrit:658242| Bumping portals to master (T128546)]] (duration: 00m 58s) |
[production] |
| 11:35 |
<elukey@cumin1001> |
END (PASS) - Cookbook sre.hadoop.change-distro-from-cdh-clients (exit_code=0) for Hadoop test cluster: Change Hadoop distribution - elukey@cumin1001 |
[production] |
| 11:33 |
<elukey@cumin1001> |
START - Cookbook sre.hadoop.change-distro-from-cdh-clients for Hadoop test cluster: Change Hadoop distribution - elukey@cumin1001 |
[production] |
| 11:11 |
<godog> |
thanos delete old orphaned blocks with replica=unset label |
[production] |
| 10:58 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host scandium.eqiad.wmnet |
[production] |
| 10:52 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.reboot-single for host scandium.eqiad.wmnet |
[production] |
| 10:44 |
<godog> |
swift decrease weight for ms-be20[16,18,20,22] - T272837 |
[production] |
| 10:20 |
<elukey> |
restart memcached on an-tool1010 to flush superset's cache |
[analytics] |
| 10:18 |
<elukey> |
restart superset to remove druid datasources support - T263972 |
[analytics] |
| 10:00 |
<moritzm> |
installing imagemagick security updates on stretch |
[production] |
| 09:57 |
<joal> |
Changing ownership of archive WMF files to analytics:analytics-privatedata-users after update of oozie jobs |
[analytics] |
| 09:55 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host urldownloader1002.wikimedia.org |
[production] |
| 09:52 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.reboot-single for host urldownloader1002.wikimedia.org |
[production] |
| 09:49 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host urldownloader2002.wikimedia.org |
[production] |
| 09:48 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.reboot-single for host urldownloader2002.wikimedia.org |
[production] |
| 09:40 |
<godog> |
bounce apache2 on logstash1024, stuck on high cpu |
[production] |
| 09:21 |
<marostegui@deploy1001> |
Synchronized wmf-config/etcd.php: Add x2 to the mapping array T269324 (duration: 00m 58s) |
[production] |
| 09:17 |
<moritzm> |
installing samba security updates on stretch |
[production] |
| 09:15 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Add x2 to the mapping array T269324 (duration: 01m 01s) |
[production] |
| 09:06 |
<ema> |
cp3054: install varnish 6.0.1-1wm2 -- 6.0.1 without https://github.com/varnishcache/varnish-cache/pull/2705 T264398 |
[production] |
| 08:47 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1142 (re)pooling @ 100%: After upgrading its kernel', diff saved to https://phabricator.wikimedia.org/P13944 and previous config saved to /var/cache/conftool/dbconfig/20210125-084715-root.json |
[production] |
| 08:32 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1142 (re)pooling @ 75%: After upgrading its kernel', diff saved to https://phabricator.wikimedia.org/P13943 and previous config saved to /var/cache/conftool/dbconfig/20210125-083211-root.json |
[production] |
| 08:17 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1142 (re)pooling @ 50%: After upgrading its kernel', diff saved to https://phabricator.wikimedia.org/P13942 and previous config saved to /var/cache/conftool/dbconfig/20210125-081708-root.json |
[production] |
| 08:02 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1142 (re)pooling @ 25%: After upgrading its kernel', diff saved to https://phabricator.wikimedia.org/P13941 and previous config saved to /var/cache/conftool/dbconfig/20210125-080204-root.json |
[production] |
| 07:33 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1142 for kernel upgrade', diff saved to https://phabricator.wikimedia.org/P13940 and previous config saved to /var/cache/conftool/dbconfig/20210125-073322-marostegui.json |
[production] |
| 06:44 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Add x2 eqiad to dbctl T269324', diff saved to https://phabricator.wikimedia.org/P13939 and previous config saved to /var/cache/conftool/dbconfig/20210125-064419-marostegui.json |
[production] |
| 06:43 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Populate x2 eqiad hosts into dbctl T269324', diff saved to https://phabricator.wikimedia.org/P13938 and previous config saved to /var/cache/conftool/dbconfig/20210125-064305-marostegui.json |
[production] |
|
2021-01-23
§
|
| 22:50 |
<Krinkle> |
restarted cvn-app8 and cvn-app9 |
[cvn] |
| 22:39 |
<Operator873> |
CVNBot15 restarted after DC and failing to regain nick |
[cvn] |
| 22:21 |
<volker-e@deploy1001> |
Finished deploy [design/style-guide@63e39e7]: Deploy design/style-guide: 63e39e7 “Components”: Amend button groups states SVG font stack (#427) (duration: 00m 06s) |
[production] |
| 22:21 |
<volker-e@deploy1001> |
Started deploy [design/style-guide@63e39e7]: Deploy design/style-guide: 63e39e7 “Components”: Amend button groups states SVG font stack (#427) |
[production] |
| 04:05 |
<ryankemper> |
Depooled `wdqs1013` (it has ~50 mins of lag to catch up on, and also the bad gateway above) |
[production] |
| 04:03 |
<ryankemper> |
Restarted `wdqs-blazegraph` on `wdqs1013`: `sudo systemctl restart wdqs-blazegraph` |
[production] |
| 01:52 |
<dzahn@cumin1001> |
conftool action : set/pooled=yes; selector: name=mw2332.codfw.wmnet |
[production] |
| 01:52 |
<dzahn@cumin1001> |
conftool action : set/pooled=yes; selector: name=mw2328.codfw.wmnet |
[production] |
| 01:51 |
<dzahn@cumin1001> |
conftool action : set/pooled=no; selector: name=mw2332.codfw.wmnet |
[production] |
| 01:51 |
<dzahn@cumin1001> |
conftool action : set/pooled=no; selector: name=mw2328.codfw.wmnet |
[production] |
| 01:50 |
<dzahn@cumin1001> |
conftool action : set/pooled=yes; selector: name=mw2330.codfw.wmnet |
[production] |