201-250 of 10000 results (30ms)
2021-07-21 §
17:10 <majavah> deploying calico v3.18.4 T280342 [tools]
16:59 <majavah> deploying calico v3.18.4 T280342 [paws]
16:49 <urbanecm> [urbanecm@mwmaint2002 ~]$ time /usr/local/bin/mw-cli-wrapper /usr/local/bin/foreachwikiindblist /srv/mediawiki/dblists/growthexperiments.dblist extensions/GrowthExperiments/maintenance/updateMenteeData.php # T285811 [production]
16:46 <dancy> restarting Gerrit to fix plugins [releng]
16:32 <lucaswerkmeister-wmde@deploy1002> Synchronized php-1.37.0-wmf.15/extensions/WikibaseLexeme/WikibaseLexeme.entitytypes.repo.php: Backport: [[gerrit:705912|Define PREFETCHING_TERM_LOOKUP for all types in client and repo (T287085)]] (2/2) – I think this might be the fastest way to fix the errors (duration: 01m 05s) [production]
16:27 <lucaswerkmeister-wmde@deploy1002> Synchronized php-1.37.0-wmf.15/extensions/WikibaseLexeme/WikibaseLexeme.entitytypes.php: restore previous state after previous scap failed on canaries with seemingly legitimate error (duration: 01m 04s) [production]
16:23 <lucaswerkmeister-wmde@deploy1002> scap failed: average error rate on 2/6 canaries increased by 10x (rerun with --force to override this check, see https://logstash.wikimedia.org/goto/83629bcb5560d11e61d3085c89dd9ed6 for details) [production]
16:19 <herron@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on logstash2001.codfw.wmnet with reason: REIMAGE [production]
16:17 <herron@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on logstash2001.codfw.wmnet with reason: REIMAGE [production]
16:07 <dancy> Updating plugins on releases-jenkins [releng]
16:04 <jbond> upload cas_6.3.2-1+wmf10u1 to apt [production]
15:52 <majavah> add my key to passwords::root::extra_keys [paws]
15:45 <volans@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 0:05:00 on 10.3.0.1,cumin1001.mgmt,debmonitor.wikimedia.org with reason: testing new feature [production]
15:45 <volans@cumin2002> START - Cookbook sre.hosts.downtime for 0:05:00 on 10.3.0.1,cumin1001.mgmt,debmonitor.wikimedia.org with reason: testing new feature [production]
15:35 <dzahn@cumin1001> conftool action : set/weight=10; selector: name=mw143[7-8].eqiad.wmnet [production]
15:34 <dzahn@cumin1001> conftool action : set/weight=10; selector: name=mw143[7-8].eqiad.wmnet,service=canary [production]
15:17 <moritzm> installing intel-microcode security updates on stretch [production]
15:11 <moritzm> installing apt bugfix updates from Buster 10.10 point release [production]
15:00 <majavah> starting kubernetes upgrades T280302 [paws]
15:00 <urbanecm> deployment-prep: Change password for `Martin Urbanec` at votewiki [releng]
14:51 <reedy@deploy1002> Finished scap: Fix some VE translation issues for T286679 (duration: 04m 45s) [production]
14:46 <reedy@deploy1002> Started scap: Fix some VE translation issues for T286679 [production]
14:40 <papaul> powerdown ms-be2038 for BBU replacement [production]
14:35 <majavah> updating systemd on toolforge stretch bastions T287036 [tools]
14:33 <dzahn@cumin1001> conftool action : set/weight=10; selector: name=mw143[7-8].eqiad.wmnet,service=canary [production]
14:19 <hashar@deploy1002> Finished deploy [gerrit/gerrit@a5c9d35]: Update its-phabricator: Urlencode POST to conduit # T280197 (duration: 00m 09s) [production]
14:19 <hashar@deploy1002> Started deploy [gerrit/gerrit@a5c9d35]: Update its-phabricator: Urlencode POST to conduit # T280197 [production]
14:07 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts mw1294.eqiad.wmnet [production]
14:07 <Jeff_Green> authdns-update to remove deprecated records related to fundraising.wikimedia.org [production]
14:01 <moritzm> imported systemd 241-5~bpo9+wmf1 to component/systemd241 T287036 [production]
13:52 <dzahn@cumin1001> START - Cookbook sre.hosts.decommission for hosts mw1294.eqiad.wmnet [production]
13:49 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts mw1293.eqiad.wmnet [production]
13:39 <dzahn@cumin1001> START - Cookbook sre.hosts.decommission for hosts mw1293.eqiad.wmnet [production]
13:20 <jiji@cumin1001> conftool action : set/pooled=true; selector: name=eqiad,dnsdisc=mwdebug [production]
13:15 <jiji@cumin1001> conftool action : set/pooled=true; selector: name=codfw,dnsdisc=mwdebug [production]
13:10 <hashar@deploy1002> Synchronized php: group1 wikis to 1.37.0-wmf.15 (duration: 01m 13s) [production]
13:09 <godog> apply https://gerrit.wikimedia.org/r/c/operations/puppet/+/705705 to puppetdb hosts [production]
13:09 <hashar@deploy1002> rebuilt and synchronized wikiversions files: group1 wikis to 1.37.0-wmf.15 [production]
13:09 <mutante> mw1293, mw1294 - formerly jobrunner canaries, depooled, replaced by new jobrunner canaries mw1437, mw1438 [production]
13:08 <dzahn@cumin1001> conftool action : set/pooled=inactive; selector: name=mw129[34].eqiad.wmnet [production]
13:04 <dzahn@cumin1001> conftool action : set/pooled=yes; selector: name=mw143[78].eqiad.wmnet [production]
12:45 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mw[1437-1438].eqiad.wmnet with reason: new host [production]
12:44 <dzahn@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on mw[1437-1438].eqiad.wmnet with reason: new host [production]
11:59 <arturo> deploying jobs-framework-api 07346d715d17585db9c16dd152cc91ef0bea33c3 (T286108) [tools]
11:58 <arturo> deploying jobs-framework-api 07346d715d17585db9c16dd152cc91ef0bea33c3 (T286108) [toolsbeta]
11:26 <hashar@deploy1002> Finished deploy [integration/docroot@0515d9c]: Support linking to individual doc.wikimedia.org tiles (duration: 00m 09s) [production]
11:26 <hashar@deploy1002> Started deploy [integration/docroot@0515d9c]: Support linking to individual doc.wikimedia.org tiles [production]
11:11 <urbanecm@deploy1002> Synchronized wmf-config/InitialiseSettings.php: d6699dae1e96b38b4fae7e8b9817d84b56d2be6c: GrowthExperiments: Add more wikis to linkrecommendation experiment (T284481) (duration: 01m 31s) [production]
11:04 <arturo> enabling TTLAfterFinished feature gate on kubeadm live configmap (T286108) [tools]
11:01 <arturo> enabling TTLAfterFinished feature gate on static pod manifests on /etc/kubernetes/manifests/kube-{apiserver,controller-manager}.yaml in all 3 control nodes (T286108) [tools]