|
2019-11-07
§
|
| 16:02 |
<mutante> |
mw2225 restart cron (T236799) |
[production] |
| 15:58 |
<reedy@deploy1001> |
Synchronized wmf-config/InitialiseSettings-labs.php: beta logging (duration: 01m 00s) |
[production] |
| 15:41 |
<elukey> |
roll restart all jvms on Hadoop Analytics Workers to pick up the new jvm |
[analytics] |
| 15:41 |
<XioNoX> |
remove BGP to AS3491 on eqiad (left the IX) |
[production] |
| 15:40 |
<elukey@cumin1001> |
START - Cookbook sre.hadoop.roll-restart-workers |
[production] |
| 14:53 |
<jbond42> |
rebuilding compiler1001 |
[production] |
| 13:50 |
<jynus@cumin1001> |
dbctl commit (dc=all): 'Repool db1080 at 50%', diff saved to https://phabricator.wikimedia.org/P9551 and previous config saved to /var/cache/conftool/dbconfig/20191107-135018-jynus.json |
[production] |
| 13:27 |
<arturo> |
deployed registry-admission-webhook and ingress-admission-controller into the new k8s cluster (T236826) |
[tools] |
| 13:01 |
<arturo> |
creating puppet prefix `tools-k8s-worker` and a couple of VMs `tools-k8s-worker-[1,2]` T236826 |
[tools] |
| 12:57 |
<arturo> |
increasing project quota T237633 |
[tools] |
| 12:47 |
<Urbanecm> |
EU SWAT done |
[production] |
| 12:47 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: 8e71601: a36ed85: GrowthExperiments: Configure testwiki for suggested edits testing + follow up patch (T237634) (duration: 00m 59s) |
[production] |
| 12:31 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: 19034af: GrowthExperiments: Configure intro links for suggested edits (T235723) (duration: 01m 00s) |
[production] |
| 12:18 |
<joal> |
Deleting stat1007:/srv/reportupdater/output/metrics/reference-previews/baseline.tsv as asked by awight |
[analytics] |
| 12:16 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: 2be3f86: [cirrus] remove cross_cluster_single_shard_search quirk (duration: 01m 02s) |
[production] |
| 12:04 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: 5253dec: Give commonswiki filemovers `suppressredirect` rights (T236348) (duration: 01m 03s) |
[production] |
| 11:57 |
<jynus@deploy1001> |
Synchronized wmf-config/db-eqiad.php: repool es1016 fully (duration: 01m 01s) |
[production] |
| 11:54 |
<jbond42> |
update puppet_version used by CI 545289 |
[production] |
| 11:54 |
<arturo> |
point `k8s.tools.eqiad1.wikimedia.cloud` to tools-k8s-haproxy-1 T236826 |
[tools] |
| 11:50 |
<jbond42> |
rebuilding compiler1002 |
[production] |
| 11:43 |
<arturo> |
create VMs `tools-k8s-haproxy-[1,2]` T236826 |
[tools] |
| 11:43 |
<arturo> |
create puppet prefix `tools-k8s-haproxy` T236826 |
[tools] |
| 11:36 |
<jynus@cumin1001> |
dbctl commit (dc=all): 'Repool db1080 at 10%', diff saved to https://phabricator.wikimedia.org/P9550 and previous config saved to /var/cache/conftool/dbconfig/20191107-113611-jynus.json |
[production] |
| 11:16 |
<jynus> |
stop and upgrade db1080 |
[production] |
| 10:58 |
<moritzm> |
installing Java security updates on kafka-main/logstash |
[production] |
| 10:50 |
<moritzm> |
installing Java security updates on wdqs/maps |
[production] |
| 10:46 |
<jynus@cumin1001> |
dbctl commit (dc=all): 'Fully depool db1080', diff saved to https://phabricator.wikimedia.org/P9549 and previous config saved to /var/cache/conftool/dbconfig/20191107-104618-jynus.json |
[production] |
| 10:28 |
<moritzm> |
upgrading mw1277-1279 servers to PHP 7.2.24 T237239 |
[production] |
| 10:27 |
<jynus@cumin1001> |
dbctl commit (dc=all): 'Reduce db1080 weight', diff saved to https://phabricator.wikimedia.org/P9548 and previous config saved to /var/cache/conftool/dbconfig/20191107-102747-jynus.json |
[production] |
| 09:53 |
<arturo> |
replacing SSL cert for star.wmflabs.org - for real this time (T237066) |
[project-proxy] |
| 09:41 |
<jynus@deploy1001> |
Synchronized wmf-config/db-eqiad.php: repool es1016 with low weight (duration: 01m 02s) |
[production] |
| 09:30 |
<jynus> |
stop and upgrade es1016 |
[production] |
| 09:18 |
<moritzm> |
installing Java security updates on aqs/druid/Hadoop |
[production] |
| 09:12 |
<jynus@deploy1001> |
Synchronized wmf-config/db-eqiad.php: depool es1016 (duration: 01m 04s) |
[production] |
| 09:03 |
<jynus> |
stop and upgrade es2012, es2014 |
[production] |
| 08:48 |
<jynus> |
stop and upgrade es2011 |
[production] |
| 08:30 |
<jynus> |
upgrade and restart db2093 |
[production] |
| 00:21 |
<XioNoX> |
enable interface damping on primary eqsin-codfw link - T236878 |
[production] |
| 00:16 |
<Reedy> |
rm /srv/mediawiki-staging.git/gc.log && git gc --aggressive |
[releng] |
| 00:08 |
<reedy@deploy1001> |
Synchronized wmf-config/InitialiseSettings-labs.php: https://gerrit.wikimedia.org/r/549227 (duration: 01m 00s) |
[production] |
| 00:00 |
<ebernhardson@deploy1001> |
Finished deploy [search/mjolnir/deploy@d2ad2da]: bulk_daemon: support ltr model uploads (duration: 04m 29s) |
[production] |
|
2019-11-06
§
|
| 23:56 |
<ebernhardson@deploy1001> |
Started deploy [search/mjolnir/deploy@d2ad2da]: bulk_daemon: support ltr model uploads |
[production] |
| 23:55 |
<ebernhardson@deploy1001> |
Finished deploy [search/mjolnir/deploy@d2ad2da]: bulk_daemon: support ltr model uploads (duration: 14m 56s) |
[production] |
| 23:40 |
<ebernhardson@deploy1001> |
Started deploy [search/mjolnir/deploy@d2ad2da]: bulk_daemon: support ltr model uploads |
[production] |
| 22:46 |
<Krinkle> |
Fix closing pattern of "setup quibble" Console Section in Jenkins to not break on composertest jobs. |
[releng] |
| 22:39 |
<bstorm_> |
upgraded repo version of toollabs-webservice in toolsbeta-stretch to 0.49 -- changes for the new k8s cluster T215531 |
[toolsbeta] |
| 22:36 |
<mdholloway> |
MachineVision: Imported Freebase to Wikidata ID mappings on commonswiki (T227349) |
[production] |
| 22:36 |
<milimetric> |
successfully restarted webrequest bundle, webrequest druid daily, and webrequest druid hourly |
[analytics] |
| 22:32 |
<bstorm_> |
added rsync::server::wrap_with_stunnel: false to tools-sge-services prefix to fix puppet |
[tools] |
| 22:30 |
<dzahn@cumin1001> |
conftool action : set/pooled=yes; selector: name=mw1290.eqiad.wmnet |
[production] |