|
2020-02-26
ยง
|
| 22:49 |
<pt1979@cumin2001> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) |
[production] |
| 22:48 |
<pt1979@cumin2001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 22:47 |
<pt1979@cumin2001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 22:44 |
<foks> |
removing one file for legal compliance |
[production] |
| 22:29 |
<bstorm_> |
deleted pod maintain-kubeusers-6d9c45f4bc-5bqq5 to deploy new image |
[tools] |
| 22:27 |
<pt1979@cumin2001> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) |
[production] |
| 22:25 |
<pt1979@cumin2001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 22:19 |
<pt1979@cumin2001> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) |
[production] |
| 22:16 |
<pt1979@cumin2001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 22:03 |
<jeh> |
powering down cloudvirt1014 for hardware maintenance |
[admin] |
| 21:51 |
<Urbanecm> |
Password reset for User:Joax (T242941) |
[production] |
| 21:45 |
<James_F> |
Docker: Publishing mediawiki-phan-testrun:0.1.0 for T226117 |
[releng] |
| 21:28 |
<mutante> |
ganeti - shutting apt2001 down again |
[production] |
| 21:17 |
<ladsgroup@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: [[gerrit:574454|Decrease the reads for term store for clients down to Q2Mio (T219123)]], take II (duration: 01m 04s) |
[production] |
| 21:15 |
<ladsgroup@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: [[gerrit:574454|Decrease the reads for term store for clients down to Q2Mio (T219123)]] (duration: 01m 04s) |
[production] |
| 21:15 |
<mutante> |
ganeti - re-starting apt2001 which is mysteriously broken and "half up" ..as in you can't ssh to it and don't get console but it does cause icinga alerts |
[production] |
| 21:06 |
<bstorm_> |
deleting loads of stuck grid jobs |
[tools] |
| 20:35 |
<ladsgroup@deploy1001> |
Synchronized php-1.35.0-wmf.21/extensions/Wikibase/lib/includes/Store/Sql/Terms: SWAT: [[gerrit:575055|Do prefetching entity ids on batches of 20 entity per query (T246159)]] (duration: 01m 04s) |
[production] |
| 20:35 |
<bstorm_> |
hard rebooting the grid master for toolsbeta |
[toolsbeta] |
| 20:27 |
<jeh> |
rebooting tools-worker-[1008,1015,1021] |
[tools] |
| 20:23 |
<wm-bot> |
<root> Reverted to legacy Kubernetes cluster |
[tools.pywikipedia] |
| 20:22 |
<wm-bot> |
<root> Reverted to legacy Kubernetes cluster |
[tools.pywikibot-testwiki] |
| 20:22 |
<wm-bot> |
<root> Reverted to legacy Kubernetes cluster |
[tools.pyshexy] |
| 20:22 |
<wm-bot> |
<root> Reverted to legacy Kubernetes cluster |
[tools.pub] |
| 20:22 |
<wm-bot> |
<root> Reverted to legacy Kubernetes cluster |
[tools.ptable] |
| 20:21 |
<wm-bot> |
<root> Reverted to legacy Kubernetes cluster |
[tools.proxies] |
| 20:21 |
<wm-bot> |
<root> Reverted to legacy Kubernetes cluster |
[tools.proneval-gsoc17] |
| 20:21 |
<wm-bot> |
<root> Reverted to legacy Kubernetes cluster |
[tools.prompter] |
| 20:21 |
<wm-bot> |
<root> Reverted to legacy Kubernetes cluster |
[tools.position-holder-history] |
| 20:20 |
<jhuneidi@deploy1001> |
Synchronized php: group1 wikis to 1.35.0-wmf.21 refs T233869 (duration: 01m 04s) |
[production] |
| 20:20 |
<wm-bot> |
<root> Reverted to legacy Kubernetes cluster |
[tools.portal-stats] |
| 20:20 |
<jeh> |
restart toolsbeta-sgegrid-shadow |
[toolsbeta] |
| 20:19 |
<jhuneidi@deploy1001> |
rebuilt and synchronized wikiversions files: group1 wikis to 1.35.0-wmf.21 refs T233869 |
[production] |
| 20:18 |
<otto@deploy1001> |
helmfile [STAGING] Ran 'apply' command on namespace 'eventstreams' for release 'production' . |
[production] |
| 20:17 |
<wm-bot> |
<root> Reverted to legacy Kubernetes cluster |
[tools.visualcategories] |
| 20:17 |
<wm-bot> |
<root> Reverted to legacy Kubernetes cluster |
[tools.video2commons-test] |
| 20:16 |
<wm-bot> |
<root> Reverted to legacy Kubernetes cluster |
[tools.video-cat-bot] |
| 20:16 |
<wm-bot> |
<root> Reverted to legacy Kubernetes cluster |
[tools.vendor] |
| 20:16 |
<wm-bot> |
<root> Reverted to legacy Kubernetes cluster |
[tools.usrd-tools] |
| 20:15 |
<wm-bot> |
<root> Reverted to legacy Kubernetes cluster |
[tools.userrank] |
| 20:15 |
<wm-bot> |
<root> Reverted to legacy Kubernetes cluster |
[tools.usernamesearch] |
| 20:15 |
<bstorm_> |
rebooting tools-sgegrid-master because it actually had the permissions thing going on still |
[tools] |
| 20:15 |
<wm-bot> |
<root> Reverted to legacy Kubernetes cluster |
[tools.user-contributions-feed] |
| 20:15 |
<wm-bot> |
<root> Reverted to legacy Kubernetes cluster |
[tools.usage] |
| 20:15 |
<wm-bot> |
<root> Reverted to legacy Kubernetes cluster |
[tools.uploadhelper-ir] |
| 20:15 |
<wm-bot> |
<root> Reverted to legacy Kubernetes cluster |
[tools.upload-stats-bot] |
| 20:14 |
<wm-bot> |
<root> Reverted to legacy Kubernetes cluster |
[tools.unpkg] |
| 20:14 |
<wm-bot> |
<root> Reverted to legacy Kubernetes cluster |
[tools.universalviewer] |
| 20:14 |
<wm-bot> |
<root> Reverted to legacy Kubernetes cluster |
[tools.typoscan] |
| 20:14 |
<wm-bot> |
<root> Reverted to legacy Kubernetes cluster |
[tools.twltools] |