|
2020-02-03
§
|
| 16:38 |
<eevans@deploy1001> |
helmfile [STAGING] Ran 'apply' command on namespace 'sessionstore' for release 'staging' . |
[production] |
| 16:28 |
<James_F> |
Zuul: Publish `npm run doc` for Vector T242674 |
[releng] |
| 15:38 |
<XioNoX> |
rollback: add debug on eqiad-knams link interfaces - T240659 |
[production] |
| 15:33 |
<XioNoX> |
add debug on eqiad-knams link interfaces - T240659 |
[production] |
| 14:59 |
<moritzm> |
restarting exim on phab* to pick up libidn security update |
[production] |
| 14:55 |
<moritzm> |
restarting superset on an-tool1004/1005 to pick up libidn security update |
[production] |
| 14:44 |
<moritzm> |
restarting apache on an-tool*. cloudmetrics*, logstash*, grafana1002 to pick up libidn security update |
[production] |
| 14:21 |
<moritzm> |
restarting slapd on ldap-corp* to pick up libidn2 security updates |
[production] |
| 14:18 |
<cdanis> |
T243634 ✔️ cdanis@cp4031.ulsfo.wmnet ~ 🕤☕ sudo varnish-frontend-restart |
[production] |
| 14:12 |
<arturo> |
move tools-prometheus-04 from cloudvirt1022 to cloudvirt1013 |
[tools] |
| 13:58 |
<moritzm> |
installing libidn2 security updates |
[production] |
| 13:32 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 13:32 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 13:32 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 13:32 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 13:31 |
<moritzm> |
rebooting ganeti1009 - ganeti1022 to pick up microcode update T228924 |
[production] |
| 12:58 |
<XioNoX> |
deactivate v6 BGP to AS25596 |
[production] |
| 12:57 |
<moritzm> |
installing spamassassin security updates |
[production] |
| 12:53 |
<Urbanecm> |
Previous message should be "EU SWAT done" |
[production] |
| 12:52 |
<Urbanecm> |
Morning SWAT done |
[production] |
| 12:52 |
<Urbanecm> |
Purge https://en.wikipedia.org/static/images/project-logos/zh_classicalwiki*.png (T243509) |
[production] |
| 12:51 |
<urbanecm@deploy1001> |
Synchronized static/images/project-logos/: SWAT: af0b745: Update logo for zh_classical wiki (T243509) (duration: 01m 06s) |
[production] |
| 12:48 |
<arturo> |
shutdown tools-prometheus-01 and tools-prometheus-02, after fixing the proxy `tools-prometheus.wmflabs.org` to tools-prometheus-03, data synced (T238096) |
[tools] |
| 12:45 |
<urbanecm@deploy1001> |
Synchronized dblists/mobilemainpagelegacy.dblist: SWAT: e9387b2: Disable MobileFrontend Mainpage special casing on frwiktionary (T241888) (duration: 01m 05s) |
[production] |
| 12:40 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: 5f13c19: Add minerva custom log for la.wiki (T240728; 2/2) (duration: 01m 06s) |
[production] |
| 12:37 |
<urbanecm@deploy1001> |
Synchronized static/images/mobile/copyright/: SWAT: 5f13c19: Add minerva custom log for la.wiki (T240728; 1/2) (duration: 01m 06s) |
[production] |
| 12:35 |
<moritzm> |
installing openjpeg2 security updates |
[production] |
| 12:31 |
<Urbanecm> |
Purge https://en.wikipedia.org/static/images/mobile/copyright/wikipedia-wordmark-szl.svg (T233104) |
[production] |
| 12:30 |
<urbanecm@deploy1001> |
Synchronized static/images/mobile/copyright/: SWAT: 76e67cd: e266e25: Add wordmarks for szlwiki and etwiki (T233104, T230379) (duration: 01m 06s) |
[production] |
| 12:29 |
<urbanecm@deploy1001> |
Synchronized static/images/mobile/copyright/: SWAT: 76e67cd: e266e25: Add static wordmarks for szlwiki and etwiki (T233104, T230379) (duration: 01m 06s) |
[production] |
| 12:25 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: 32e0356: Add vzg-easydb.gbv.de to the wgCopyUploadsDomains (T243118) (duration: 01m 07s) |
[production] |
| 12:20 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: 6c48af8: Assign editautopatrolprotected to hewiki patrollers (T243665) (duration: 01m 06s) |
[production] |
| 12:14 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: 6b497e7: Wikidata - enable TaintedRefs (T241989) (duration: 01m 06s) |
[production] |
| 12:09 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: 0c0ef87: Add wgImportSources for hiwikibooks (T244022) (duration: 01m 05s) |
[production] |
| 12:07 |
<urbanecm@deploy1001> |
Synchronized wmf-config/CommonSettings.php: SWAT: Remove $wgImgAuthDetails=true (T153459) (duration: 01m 36s) |
[production] |
| 11:38 |
<ema> |
powercycle cp3057 T244127 T238305 |
[production] |
| 11:20 |
<elukey> |
restart oozie on an-coord1001 |
[analytics] |
| 10:24 |
<godog> |
temp disable puppet on cp hosts as precaution for https://gerrit.wikimedia.org/r/c/operations/puppet/+/563977 |
[production] |
| 10:11 |
<elukey> |
enable all timers on an-coord1001 after spark encryption/auth settings |
[analytics] |
| 10:08 |
<moritzm> |
installing sudo security updates on stretch |
[production] |
| 09:57 |
<arturo> |
installed virtuoso-opensource-7 on the etytree-a server by hand (see T244111) |
[etytree] |
| 09:38 |
<arturo> |
tools-prometheus-01: systemctl stop prometheus@tools. Another try to migrate data to tools-prometheus-{03,04} (T238096) |
[tools] |
| 09:32 |
<elukey> |
roll restart yarn node managers again to pick up spark encryption/authentication settings |
[analytics] |
| 08:34 |
<elukey> |
stop timers on an-coord1001 to drain the cluster and ease the deploy of spark encryption settings |
[analytics] |
| 07:58 |
<elukey> |
roll restart hadoop yarn node managers to pick up new libcrypto.so link (shouldn't be necessary but just in case) |
[analytics] |
| 07:24 |
<elukey> |
create /usr/lib/x86_64-linux-gnu/libcrypto.so on all the analytics nodes via puppet |
[analytics] |
| 02:22 |
<Cam11598> |
Restarted CVNBot15 and Cubbie |
[cvn] |
| 02:19 |
<bd808> |
Hard restart of webservice to pick up redirect logic (T242719) |
[tools.docker-registry] |