|
2019-11-07
§
|
| 15:41 |
<XioNoX> |
remove BGP to AS3491 on eqiad (left the IX) |
[production] |
| 15:40 |
<elukey@cumin1001> |
START - Cookbook sre.hadoop.roll-restart-workers |
[production] |
| 14:53 |
<jbond42> |
rebuilding compiler1001 |
[production] |
| 13:50 |
<jynus@cumin1001> |
dbctl commit (dc=all): 'Repool db1080 at 50%', diff saved to https://phabricator.wikimedia.org/P9551 and previous config saved to /var/cache/conftool/dbconfig/20191107-135018-jynus.json |
[production] |
| 12:47 |
<Urbanecm> |
EU SWAT done |
[production] |
| 12:47 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: 8e71601: a36ed85: GrowthExperiments: Configure testwiki for suggested edits testing + follow up patch (T237634) (duration: 00m 59s) |
[production] |
| 12:31 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: 19034af: GrowthExperiments: Configure intro links for suggested edits (T235723) (duration: 01m 00s) |
[production] |
| 12:16 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: 2be3f86: [cirrus] remove cross_cluster_single_shard_search quirk (duration: 01m 02s) |
[production] |
| 12:04 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: 5253dec: Give commonswiki filemovers `suppressredirect` rights (T236348) (duration: 01m 03s) |
[production] |
| 11:57 |
<jynus@deploy1001> |
Synchronized wmf-config/db-eqiad.php: repool es1016 fully (duration: 01m 01s) |
[production] |
| 11:54 |
<jbond42> |
update puppet_version used by CI 545289 |
[production] |
| 11:50 |
<jbond42> |
rebuilding compiler1002 |
[production] |
| 11:36 |
<jynus@cumin1001> |
dbctl commit (dc=all): 'Repool db1080 at 10%', diff saved to https://phabricator.wikimedia.org/P9550 and previous config saved to /var/cache/conftool/dbconfig/20191107-113611-jynus.json |
[production] |
| 11:16 |
<jynus> |
stop and upgrade db1080 |
[production] |
| 10:58 |
<moritzm> |
installing Java security updates on kafka-main/logstash |
[production] |
| 10:50 |
<moritzm> |
installing Java security updates on wdqs/maps |
[production] |
| 10:46 |
<jynus@cumin1001> |
dbctl commit (dc=all): 'Fully depool db1080', diff saved to https://phabricator.wikimedia.org/P9549 and previous config saved to /var/cache/conftool/dbconfig/20191107-104618-jynus.json |
[production] |
| 10:28 |
<moritzm> |
upgrading mw1277-1279 servers to PHP 7.2.24 T237239 |
[production] |
| 10:27 |
<jynus@cumin1001> |
dbctl commit (dc=all): 'Reduce db1080 weight', diff saved to https://phabricator.wikimedia.org/P9548 and previous config saved to /var/cache/conftool/dbconfig/20191107-102747-jynus.json |
[production] |
| 09:41 |
<jynus@deploy1001> |
Synchronized wmf-config/db-eqiad.php: repool es1016 with low weight (duration: 01m 02s) |
[production] |
| 09:30 |
<jynus> |
stop and upgrade es1016 |
[production] |
| 09:18 |
<moritzm> |
installing Java security updates on aqs/druid/Hadoop |
[production] |
| 09:12 |
<jynus@deploy1001> |
Synchronized wmf-config/db-eqiad.php: depool es1016 (duration: 01m 04s) |
[production] |
| 09:03 |
<jynus> |
stop and upgrade es2012, es2014 |
[production] |
| 08:48 |
<jynus> |
stop and upgrade es2011 |
[production] |
| 08:30 |
<jynus> |
upgrade and restart db2093 |
[production] |
| 00:21 |
<XioNoX> |
enable interface damping on primary eqsin-codfw link - T236878 |
[production] |
| 00:08 |
<reedy@deploy1001> |
Synchronized wmf-config/InitialiseSettings-labs.php: https://gerrit.wikimedia.org/r/549227 (duration: 01m 00s) |
[production] |
| 00:00 |
<ebernhardson@deploy1001> |
Finished deploy [search/mjolnir/deploy@d2ad2da]: bulk_daemon: support ltr model uploads (duration: 04m 29s) |
[production] |
|
2019-11-06
§
|
| 23:56 |
<ebernhardson@deploy1001> |
Started deploy [search/mjolnir/deploy@d2ad2da]: bulk_daemon: support ltr model uploads |
[production] |
| 23:55 |
<ebernhardson@deploy1001> |
Finished deploy [search/mjolnir/deploy@d2ad2da]: bulk_daemon: support ltr model uploads (duration: 14m 56s) |
[production] |
| 23:40 |
<ebernhardson@deploy1001> |
Started deploy [search/mjolnir/deploy@d2ad2da]: bulk_daemon: support ltr model uploads |
[production] |
| 22:36 |
<mdholloway> |
MachineVision: Imported Freebase to Wikidata ID mappings on commonswiki (T227349) |
[production] |
| 22:30 |
<dzahn@cumin1001> |
conftool action : set/pooled=yes; selector: name=mw1290.eqiad.wmnet |
[production] |
| 22:29 |
<mholloway-shell@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: Enable MachineVision on commonswiki (T227349) (duration: 01m 00s) |
[production] |
| 22:24 |
<mholloway-shell@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: MachineVision: Delay annotation jobs on commonswiki only (duration: 01m 01s) |
[production] |
| 22:17 |
<mdholloway> |
created MachineVision extension tables on commonswiki |
[production] |
| 22:13 |
<XioNoX> |
push standard forwarding-options to cr3/4-ulsfo |
[production] |
| 22:04 |
<dzahn@cumin1001> |
conftool action : set/pooled=no; selector: name=mw1290.eqiad.wmnet |
[production] |
| 22:04 |
<mholloway-shell@deploy1001> |
Synchronized private/PrivateSettings.php: Configure Google Cloud Vision API credentials (2/2) (T236426) (duration: 00m 59s) |
[production] |
| 22:04 |
<dzahn@cumin1001> |
conftool action : set/pooled=yes; selector: name=mw1247.eqiad.wmnet |
[production] |
| 22:03 |
<mholloway-shell@deploy1001> |
Synchronized private/GoogleCloudVision.php: Configure Google Cloud Vision API credentials (1/2) (T236426) (duration: 00m 59s) |
[production] |
| 21:57 |
<mholloway-shell@deploy1001> |
Synchronized php-1.35.0-wmf.5/extensions/MachineVision: Allow specifying API credentials as an associative array (T236426) (duration: 01m 01s) |
[production] |
| 21:53 |
<thcipriani> |
checkout /srv/mediawiki-staging/php-1.35.0-wmf.5/maintenance/Maintenance.php looks like a local change for debugging left behind |
[production] |
| 21:47 |
<arlolra> |
Updated Parsoid to 1d283ed (T237104, T227209, T236865) |
[production] |
| 21:35 |
<arlolra@deploy1001> |
Finished deploy [parsoid/deploy@7e86f83]: Updating Parsoid to 1d283ed (duration: 10m 22s) |
[production] |
| 21:24 |
<arlolra@deploy1001> |
Started deploy [parsoid/deploy@7e86f83]: Updating Parsoid to 1d283ed |
[production] |
| 21:21 |
<dzahn@cumin1001> |
conftool action : set/pooled=no; selector: name=mw1247.eqiad.wmnet |
[production] |
| 21:14 |
<XioNoX> |
push standard forwarding-options to cr3-esams |
[production] |
| 21:12 |
<milimetric@deploy1001> |
Finished deploy [analytics/refinery@dc85f9d]: Hdfs Cleaner and TLS columns (duration: 10m 52s) |
[production] |