|
2020-01-06
§
|
| 11:41 |
<jdrewniak@deploy1001> |
Synchronized portals/wikipedia.org/assets: Wikimedia Portals Update: [[gerrit:562236| Bumping portals to master (T128546)]] (duration: 00m 55s) |
[production] |
| 10:56 |
<urbanecm@deploy1001> |
Synchronized private/PrivateSettings.php: Revert T227416 mitigations (duration: 01m 05s) |
[production] |
| 10:39 |
<moritzm> |
installing libbsd security updates |
[production] |
| 09:53 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 09:51 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 09:32 |
<moritzm> |
reimaging mw2282 to validate correctness of apt::package_from_component for fresh installs |
[production] |
| 07:37 |
<Amir1> |
ladsgroup@mwmaint1002:~$ mwscript extensions/Wikibase/repo/maintenance/rebuildItemTerms.php --wiki=wikidatawiki --batch-size=100 --sleep=2 --file=/tmp/1mio.lines (T219301) |
[production] |
| 03:53 |
<Amir1> |
ladsgroup@mwmaint1002:~$ mwscript extensions/Wikibase/repo/maintenance/rebuildItemTerms.php --wiki=wikidatawiki --batch-size=100 --sleep=2 --file=/tmp/100k.lines (T219301) |
[production] |
| 00:06 |
<effie> |
pool cp3065 T238305 |
[production] |
| 00:05 |
<jiji@cumin1001> |
conftool action : set/pooled=yes; selector: name=cp3065.esams.wmnet |
[production] |
|
2020-01-04
§
|
| 18:11 |
<bd808> |
Shutdown tools-worker-1029 |
[tools] |
| 18:10 |
<bd808> |
kubectl delete node tools-worker-1029.tools.eqiad.wmflabs |
[tools] |
| 18:06 |
<bd808> |
Removed tools-worker-1029.tools.eqiad.wmflabs from k8s::worker_hosts hiera in preparation for decom |
[tools] |
| 16:54 |
<bstorm_> |
moving VMs tools-worker-1012/1028/1005 from cloudvirt1024 to cloudvirt1003 due to hardware errors T241884 |
[tools] |
| 16:47 |
<bstorm_> |
moving VM tools-flannel-etcd-02 from cloudvirt1024 to cloudvirt1003 due to hardware errors T241884 |
[tools] |
| 16:34 |
<arturo> |
icinga downtime cloudvirt1024 for 2 months because hardware errors (T241884) |
[admin] |
| 16:34 |
<aborrero@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 16:34 |
<aborrero@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 16:34 |
<aborrero@cumin1001> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) |
[production] |
| 16:34 |
<aborrero@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 16:16 |
<bd808> |
Draining tools-worker-10{05,12,28} due to hardware errors (T241884) |
[tools] |
| 16:13 |
<arturo> |
moving VM tools-sgewebgrid-lighttpd-0927 from cloudvirt1024 to cloudvirt1009 due to hardware errors (T241884) |
[tools] |
| 16:11 |
<arturo> |
moving VM tools-sgewebgrid-lighttpd-0926 from cloudvirt1024 to cloudvirt1009 due to hardware errors (T241884) |
[tools] |
| 16:09 |
<arturo> |
moving VM tools-sgewebgrid-lighttpd-0925 from cloudvirt1024 to cloudvirt1009 due to hardware errors (T241884) |
[tools] |
| 16:09 |
<bstorm_> |
moving VMs encoding04 and encoding05 from cloudvirt1024 to cloudvirt1003 due to hardware errors T241884 |
[video] |
| 16:08 |
<arturo> |
moving VM tools-sgewebgrid-lighttpd-0924 from cloudvirt1024 to cloudvirt1009 due to hardware errors (T241884) |
[tools] |
| 16:07 |
<arturo> |
moving VM tools-sgewebgrid-lighttpd-0923 from cloudvirt1024 to cloudvirt1009 due to hardware errors (T241884) |
[tools] |
| 16:06 |
<arturo> |
moving VM tools-sgewebgrid-lighttpd-0909 from cloudvirt1024 to cloudvirt1009 due to hardware errors (T241884) |
[tools] |
| 16:05 |
<bstorm_> |
moving VM meza-full from cloudvirt1024 to cloudvirt1003 due to hardware error T241884 |
[meza] |
| 16:04 |
<arturo> |
moving VM tools-sgeexec-0923 from cloudvirt1024 to cloudvirt1009 due to hardware errors (T241884) |
[tools] |
| 16:02 |
<arturo> |
moving VM tools-sgeexec-0910 from cloudvirt1024 to cloudvirt1009 due to hardware errors (T241873) |
[tools] |
| 16:01 |
<bstorm_> |
moving vm puppetmaster-1001 from cloudvirt1024 to cloudvirt1009 due to hardware error T241884 |
[devtools] |
| 15:59 |
<arturo> |
moving VM cyberbot-db-01 from cloudvirt1024 to cloudvirt1009 due to hardware errors (T241873) |
[cyberbot] |
| 02:00 |
<bd808> |
Moving to new Kubernetes cluster |
[tools.jouncebot] |
| 01:17 |
<bd808> |
Moving to new kubernetes cluster |
[tools.os-deprecation] |
| 01:09 |
<bd808> |
kubectl scale --replicas=3 deployment/fourohfour |
[tools.fourohfour] |
| 01:06 |
<bd808> |
Moving to new kubernetes cluster |
[tools.admin-beta] |
|
2020-01-03
§
|
| 23:33 |
<bd808> |
Moving to new kubernetes cluster and py37 |
[tools.csp-report] |
| 22:37 |
<mutante> |
- sudo vi /srv/deployment/phabricator/deployment-cache/.config on both phabricator instances to fix deployment server (remove deployment-tin (!)) |
[devtools] |
| 22:29 |
<bd808> |
Migrating to new kubernetes cluster |
[tools.openstack-browser-dev] |
| 22:24 |
<bd808> |
Migrating to new kubernetes cluster |
[tools.openstack-browser] |
| 22:22 |
<bd808> |
Migrating to new kubernetes cluster |
[tools.static] |
| 22:20 |
<bd808> |
Migrating to new kubernetes cluster |
[tools.versions] |
| 22:18 |
<bd808> |
Migrating to new kubernetes cluster |
[tools.trusty-tools] |
| 22:15 |
<bd808> |
Migrating to new kubernetes cluster |
[tools.toolviews] |
| 22:14 |
<volker-e@deploy1001> |
Finished deploy [design/style-guide@8054026]: Deploy design/style-guide: (duration: 00m 08s) |
[production] |
| 22:14 |
<volker-e@deploy1001> |
Started deploy [design/style-guide@8054026]: Deploy design/style-guide: |
[production] |