|
2020-07-03
§
|
| 08:04 |
<jayme> |
authdns-update for chartmuseum - T256970 |
[production] |
| 08:03 |
<elukey@cumin1001> |
END (FAIL) - Cookbook sre.dns.netbox (exit_code=99) |
[production] |
| 07:55 |
<moritzm> |
installing mutt security updates for jessie (stretch/buster already fixed) |
[production] |
| 07:44 |
<elukey@cumin1001> |
START - Cookbook sre.dns.netbox |
[production] |
| 07:40 |
<elukey@cumin1001> |
END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) |
[production] |
| 07:39 |
<elukey@cumin1001> |
START - Cookbook sre.hosts.decommission |
[production] |
| 07:00 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 07:00 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 06:47 |
<moritzm> |
installing php5 security updates |
[production] |
| 06:27 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 06:27 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 06:09 |
<moritzm> |
rebooting mw1390-mw1419 for kernel security updates |
[production] |
| 05:46 |
<XioNoX> |
remove chassis redundancy failover from fasw-c-eqiad for consistency with all other VCs |
[production] |
| 05:33 |
<XioNoX> |
remove chassis redundancy failover from fasw-c-codfw for consistency with all other VCs |
[production] |
|
2020-07-02
§
|
| 23:22 |
<jhuneidi@deploy1001> |
helmfile [EQIAD] Ran 'sync' command on namespace 'blubberoid' for release 'production' . |
[production] |
| 23:16 |
<jhuneidi@deploy1001> |
helmfile [CODFW] Ran 'sync' command on namespace 'blubberoid' for release 'production' . |
[production] |
| 22:03 |
<jhuneidi@deploy1001> |
helmfile [STAGING] Ran 'sync' command on namespace 'blubberoid' for release 'staging' . |
[production] |
| 21:56 |
<mutante> |
gerrit1001 (prod gerrit) - restarting gerrit service |
[production] |
| 21:52 |
<maryum> |
frwikibooks reindex sucessful, continuing on with remainder of french wikis |
[production] |
| 21:32 |
<mutante> |
gerrit - deleted gerrit db_pass from prod private repo, running puppet |
[production] |
| 21:25 |
<mutante> |
gerrit2001 - restarted gerrit |
[production] |
| 21:14 |
<mutante> |
gerrit1002 restarted gerrit |
[production] |
| 20:20 |
<maryum> |
reindexing frwikibooks to test https://gerrit.wikimedia.org/r/c/mediawiki/extensions/CirrusSearch/+/604221 |
[production] |
| 19:52 |
<mutante> |
gerrit2001 - restarting gerrit after removing db_pass from config |
[production] |
| 16:05 |
<cmjohnson@cumin1001> |
END (FAIL) - Cookbook sre.dns.netbox (exit_code=99) |
[production] |
| 16:01 |
<cmjohnson@cumin1001> |
START - Cookbook sre.dns.netbox |
[production] |
| 15:40 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) |
[production] |
| 15:37 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.reboot-single |
[production] |
| 15:23 |
<cmjohnson@cumin1001> |
END (FAIL) - Cookbook sre.dns.netbox (exit_code=99) |
[production] |
| 15:19 |
<cmjohnson@cumin1001> |
START - Cookbook sre.dns.netbox |
[production] |
| 15:07 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 15:07 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 14:42 |
<moritzm> |
rebooting mw1370-mw1389 for kernel security updates |
[production] |
| 14:33 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 14:33 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 14:03 |
<kormat> |
stopped mariadb@s8 on dbstore1005 for data restoration T256966 |
[production] |
| 12:43 |
<vgutierrez@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 12:43 |
<vgutierrez@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 12:36 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 12:36 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 12:32 |
<vgutierrez@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 12:32 |
<vgutierrez@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 12:31 |
<moritzm> |
rebooting mw1349-mw1369 for kernel security updates |
[production] |
| 12:28 |
<vgutierrez@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 12:27 |
<vgutierrez@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 12:27 |
<vgutierrez> |
rolling restart of esams load balancers to catch up on kernel upgrades |
[production] |
| 12:12 |
<XioNoX> |
pre-configure asw2-b-eqiad<->cloudsw1-c8-eqiad - T251632 |
[production] |
| 12:10 |
<vgutierrez@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 12:10 |
<vgutierrez@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 11:53 |
<vgutierrez@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |