|
2019-07-08
§
|
| 14:57 |
<marostegui> |
Failover x1 codfw from db2045 to db2069 |
[production] |
| 14:48 |
<ppchelko@deploy1001> |
Finished deploy [restbase/deploy@9a99b17]: Loosen etag regex for talk endpoint and fix alert (duration: 16m 07s) |
[production] |
| 14:44 |
<marostegui> |
Restart MySQL on db1132 to enable performance_schema - T226952 |
[production] |
| 14:43 |
<urandom> |
decommissioning restbase1017-c -- T222960 |
[production] |
| 14:32 |
<ppchelko@deploy1001> |
Started deploy [restbase/deploy@9a99b17]: Loosen etag regex for talk endpoint and fix alert |
[production] |
| 14:21 |
<papaul> |
shutting down elastic2054 for troubleshooting |
[production] |
| 14:05 |
<ppchelko@deploy1001> |
Finished deploy [restbase/deploy@8e81e98]: Release 1.0, expose talk endpoints T225733, suggestions endpoints T224754, fix summary purging T226983 (duration: 16m 11s) |
[production] |
| 14:03 |
<eevans@deploy1001> |
scap-helm sessionstore finished |
[production] |
| 14:03 |
<eevans@deploy1001> |
scap-helm sessionstore cluster staging completed |
[production] |
| 14:03 |
<eevans@deploy1001> |
scap-helm sessionstore upgrade staging -f sessionstore-staging-values.yaml stable/kask [namespace: sessionstore, clusters: staging] |
[production] |
| 13:53 |
<godog> |
reprepro --delete clearvanished on install1002 to cleanup trusty |
[production] |
| 13:52 |
<elukey> |
import AMD ROCm's Debian repo key (9386B48A1A693C5C) manually on install1002 - T224723 |
[production] |
| 13:51 |
<moritzm> |
running "apt-get --allow-releaseinfo-update" on all buster hosts which were installed prior to the final buster release |
[production] |
| 13:48 |
<ppchelko@deploy1001> |
Started deploy [restbase/deploy@8e81e98]: Release 1.0, expose talk endpoints T225733, suggestions endpoints T224754, fix summary purging T226983 |
[production] |
| 13:30 |
<godog> |
bounce prometheus@k8s on prometheus1003 |
[production] |
| 12:52 |
<godog> |
copy mtail to buster-wikimedia - T225604 |
[production] |
| 12:42 |
<kartik@deploy1001> |
scap-helm cxserver finished |
[production] |
| 12:42 |
<kartik@deploy1001> |
scap-helm cxserver cluster eqiad completed |
[production] |
| 12:42 |
<kartik@deploy1001> |
scap-helm cxserver upgrade -f cxserver-eqiad-values.yaml production stable/cxserver [namespace: cxserver, clusters: eqiad] |
[production] |
| 12:39 |
<kartik@deploy1001> |
scap-helm cxserver finished |
[production] |
| 12:39 |
<kartik@deploy1001> |
scap-helm cxserver cluster codfw completed |
[production] |
| 12:39 |
<kartik@deploy1001> |
scap-helm cxserver upgrade -f cxserver-codfw-values.yaml production stable/cxserver [namespace: cxserver, clusters: codfw] |
[production] |
| 12:36 |
<kartik@deploy1001> |
scap-helm cxserver finished |
[production] |
| 12:36 |
<kartik@deploy1001> |
scap-helm cxserver cluster staging completed |
[production] |
| 12:36 |
<kartik@deploy1001> |
scap-helm cxserver upgrade -f cxserver-staging-values.yaml staging stable/cxserver [namespace: cxserver, clusters: staging] |
[production] |
| 11:47 |
<Urbanecm> |
EU SWAT done |
[production] |
| 11:44 |
<urbanecm@deploy1001> |
Synchronized php-1.34.0-wmf.11/includes/Title.php: SWAT: [[:gerrit:521253|Title: ensure getBaseTitle and getRootTitle return valid Titles]] (T225585) (duration: 00m 50s) |
[production] |
| 11:39 |
<Urbanecm> |
Purged 14 logo urls for T227418 |
[production] |
| 11:36 |
<urbanecm@deploy1001> |
Synchronized wmf-config/CirrusSearch-common.php: SWAT: [[:gerrit:521038|Fix array shape for $wgCirrusSearchExtraIndexes]] (T227379) (duration: 00m 51s) |
[production] |
| 11:31 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: [[:gerrit:521183|Remove HD logos for projects with no entry in wgLogo or add a wgLogo entry]] (2/2, T227418) (duration: 00m 49s) |
[production] |
| 11:30 |
<urbanecm@deploy1001> |
Synchronized static/images/project-logos/: SWAT: [[:gerrit:521183|Remove HD logos for projects with no entry in wgLogo or add a wgLogo entry]] (1/2, T227418) (duration: 00m 49s) |
[production] |
| 11:26 |
<moritzm> |
installing poolcounter1004/1005 |
[production] |
| 11:25 |
<urbanecm@deploy1001> |
Synchronized php-1.34.0-wmf.11/extensions/AbuseFilter/: SWAT: [[:gerrit:520991|Fix query in normalizeThrottleParameters]] (T209565) (duration: 00m 51s) |
[production] |
| 11:22 |
<ladsgroup@deploy1001> |
Synchronized wmf-config/Wikibase.php: [[gerrit:520780|Disable Wikidata for ProofreadPage namespaces (T227201)]] (duration: 00m 50s) |
[production] |
| 11:16 |
<ladsgroup@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: [[gerrit:521221|Enable jsonld output format for wikibase entities everywhere (T207168)]] (duration: 00m 49s) |
[production] |
| 11:11 |
<urbanecm@deploy1001> |
Synchronized wmf-config/flaggedrevs.php: SWAT: [[:gerrit:521194|Remove "עמוד" namespace from wgFlaggedRevsNamespaces for hewikisource]] (T227000) (duration: 00m 49s) |
[production] |
| 11:09 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: [[:gerrit:520997|Add several Ukrainian government websites to wgCopyUploadsDomains]] (T227366) (duration: 00m 49s) |
[production] |
| 11:07 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: [[:gerrit:520507|Create "autopatrolled" user group on az.wiktionary]] (T227208) (duration: 00m 49s) |
[production] |
| 11:04 |
<urbanecm@deploy1001> |
Synchronized dblists/commonsuploads.dblist: SWAT: [[:gerrit:520507|Create "autopatrolled" user group on az.wiktionary]] (T227208) (duration: 00m 50s) |
[production] |
| 10:56 |
<moritzm> |
installing poolcounter2003/2004 |
[production] |
| 10:39 |
<jdrewniak@deploy1001> |
Synchronized portals: Wikimedia Portals Update: [[gerrit:521246| Bumping portals to master (T128546)]] (duration: 00m 49s) |
[production] |
| 10:38 |
<jdrewniak@deploy1001> |
Synchronized portals/wikipedia.org/assets: Wikimedia Portals Update: [[gerrit:521246| Bumping portals to master (T128546)]] (duration: 00m 51s) |
[production] |
| 09:51 |
<elukey@cumin1001> |
END (FAIL) - Cookbook sre.ganeti.makevm (exit_code=99) |
[production] |
| 09:51 |
<elukey@cumin1001> |
START - Cookbook sre.ganeti.makevm |
[production] |
| 09:49 |
<ema> |
removed /srv/prometheus/ops/targets/varnish-upload-ats_mtail_$DC.yaml from prometheus hosts |
[production] |
| 08:27 |
<moritzm> |
updated buster installer images to final release |
[production] |
| 07:43 |
<moritzm> |
rebooting hassium to pick up MDS-enabled qemu |
[production] |
| 07:43 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 07:43 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 07:40 |
<moritzm> |
rebooting weblog1001 for kernel security update |
[production] |