|
2020-07-02
§
|
| 23:22 |
<jhuneidi@deploy1001> |
helmfile [EQIAD] Ran 'sync' command on namespace 'blubberoid' for release 'production' . |
[production] |
| 23:16 |
<jhuneidi@deploy1001> |
helmfile [CODFW] Ran 'sync' command on namespace 'blubberoid' for release 'production' . |
[production] |
| 22:03 |
<jhuneidi@deploy1001> |
helmfile [STAGING] Ran 'sync' command on namespace 'blubberoid' for release 'staging' . |
[production] |
| 21:56 |
<mutante> |
gerrit1001 (prod gerrit) - restarting gerrit service |
[production] |
| 21:52 |
<maryum> |
frwikibooks reindex sucessful, continuing on with remainder of french wikis |
[production] |
| 21:32 |
<mutante> |
gerrit - deleted gerrit db_pass from prod private repo, running puppet |
[production] |
| 21:25 |
<mutante> |
gerrit2001 - restarted gerrit |
[production] |
| 21:14 |
<mutante> |
gerrit1002 restarted gerrit |
[production] |
| 20:20 |
<maryum> |
reindexing frwikibooks to test https://gerrit.wikimedia.org/r/c/mediawiki/extensions/CirrusSearch/+/604221 |
[production] |
| 19:52 |
<mutante> |
gerrit2001 - restarting gerrit after removing db_pass from config |
[production] |
| 19:26 |
<James_F> |
Zuul: [mediawiki/extensions/CheckUser] Add EventLogging to phan deps |
[releng] |
| 18:16 |
<joal> |
Launch a manual instance of mediawiki-history-denormalize to release data despite oozie failing |
[analytics] |
| 16:17 |
<joal> |
rerun mediawiki-history-denormalize-wf-2020-06 after oozie sharelib bump through manual restart |
[analytics] |
| 16:05 |
<cmjohnson@cumin1001> |
END (FAIL) - Cookbook sre.dns.netbox (exit_code=99) |
[production] |
| 16:03 |
<Urbanecm> |
Restart StewardBot |
[tools.stewardbots] |
| 16:03 |
<Urbanecm> |
Increase the waiting time before joining channels (bot can't join #wikimedia-stewards) |
[tools.stewardbots] |
| 16:01 |
<cmjohnson@cumin1001> |
START - Cookbook sre.dns.netbox |
[production] |
| 16:00 |
<joal> |
rerun mediawiki-history-denormalize-wf-2020-06 after oozie sharelib bump through manual restart |
[releng] |
| 15:58 |
<arturo> |
created VM 'builder-envoy-02' (T256983) with ceph storage |
[packaging] |
| 15:56 |
<arturo> |
bump RAM quota to make room for new envoy builder VM (T256983) |
[packaging] |
| 15:55 |
<Urbanecm> |
Restart StewardBot |
[tools.stewardbots] |
| 15:45 |
<MacFan4000> |
restart due to downtime |
[tools.zppixbot] |
| 15:41 |
<arturo> |
`sudo wmcs-openstack --os-compute-api-version 2.55 flavor create --private --vcpus 8 --disk 300 --ram 16384 --property aggregate_instance_extra_specs:ceph=true --description "for packaging envoy" bigdisk-ceph` (T256983) |
[admin] |
| 15:40 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) |
[production] |
| 15:37 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.reboot-single |
[production] |
| 15:30 |
<Urbanecm> |
Restart StewardBot |
[tools.stewardbots] |
| 15:23 |
<cmjohnson@cumin1001> |
END (FAIL) - Cookbook sre.dns.netbox (exit_code=99) |
[production] |
| 15:19 |
<cmjohnson@cumin1001> |
START - Cookbook sre.dns.netbox |
[production] |
| 15:07 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 15:07 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 14:42 |
<moritzm> |
rebooting mw1370-mw1389 for kernel security updates |
[production] |
| 14:33 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 14:33 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 14:03 |
<kormat> |
stopped mariadb@s8 on dbstore1005 for data restoration T256966 |
[production] |
| 12:43 |
<vgutierrez@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 12:43 |
<vgutierrez@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 12:41 |
<joal> |
retry mediawiki-history-denormalize-wf-2020-06 |
[analytics] |
| 12:36 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 12:36 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 12:32 |
<vgutierrez@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 12:32 |
<vgutierrez@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 12:31 |
<moritzm> |
rebooting mw1349-mw1369 for kernel security updates |
[production] |
| 12:28 |
<vgutierrez@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 12:27 |
<vgutierrez@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 12:27 |
<vgutierrez> |
rolling restart of esams load balancers to catch up on kernel upgrades |
[production] |