1251-1300 of 10000 results (21ms)
2020-11-12 §
14:33 <hashar> Beta cluster is available again. The frontend cache had a mixup between varnish 5 running and config being for varnish 6. The package got upgraded and it seems to serve traffic now. # T267561 [releng]
11:19 <hashar> Rename https://github.com/wikimedia/mediawiki-services-mathjax-attic back to https://github.com/wikimedia/mediawiki-services-mathjax and keeping it unarchived for Gerrit to replicate to it # T249908 [releng]
11:14 <hashar> Unarchiving https://github.com/wikimedia/mediawiki-services-mathjax and renaming it to https://github.com/wikimedia/mediawiki-services-mathjax-attic # T249908 [releng]
08:29 <hashar> Building containers for composer update from 1.10.5 .. 1.10.17 : https://gerrit.wikimedia.org/r/c/integration/config/+/640574 [releng]
2020-11-11 §
09:50 <awight> metawiki: Promoting User:Jan Dittrich (WMDE) into centralnoticeadmin... [releng]
2020-11-10 §
23:19 <dpifke> Cherry-picking https://gerrit.wikimedia.org/r/c/operations/puppet/+/640226 and https://gerrit.wikimedia.org/r/c/performance/coal/+/640227 in beta; should only affect deployment-webperf11. [releng]
22:13 <thcipriani> restarting php7.2-fpm on deployment-jobrunner03 and deployment-parsoid11 [releng]
18:40 <apergos> deployment-prep: switched deployment-prep to libicu63, see T264991 [releng]
2020-11-09 §
22:18 <awight> metawiki: Promoting User:Jan Dittrich into centralnoticeadmin... [releng]
14:18 <hashar> deployment-prep: removing role::beta::availability_collector from deployment-cache06 since that broke puppet. The role no more exists # T267006 [releng]
2020-11-07 §
00:16 <dpifke> Cherry-picking https://gerrit.wikimedia.org/r/c/operations/puppet/+/639885 and https://gerrit.wikimedia.org/r/c/performance/arc-lamp/+/639886 in beta. Should only affect deployment-webperf12. [releng]
2020-11-05 §
18:38 <Reedy> beta has been stuck for ~22H. Killed backlog of jobs, seems to be running now [releng]
09:19 <elukey> fixed kafka broker names (s/.eqiad.wmflabs/.eqiad1.wikimedia.cloud/g) in deployment-prep's hiera global config to unblock puppet [releng]
2020-11-04 §
21:33 <hashar> devtools upgraded gerrit da40615f726..9275b308ffd [releng]
20:33 <hashar> devtools: fixed java version on gerrit-prod-1001.devtools.eqiad.wmflabs by setting java::profile hieradata variable at https://horizon.wikimedia.org/project/instances/48c5ff1c-2885-410d-beeb-d5a57a0a91c7/ [releng]
19:59 <hashar> operations/software/gerrit : pushed upstream git tag, synced the master branch # T262241 [releng]
18:50 <dpifke> Re-cherry-picked https://gerrit.wikimedia.org/r/c/operations/puppet/+/639216 in beta. [releng]
18:05 <dpifke> Cherry-picking https://gerrit.wikimedia.org/r/c/analytics/statsv/+/639223 in beta. [releng]
17:50 <dpifke> Reverting previous cherry-pick for now; requires minor code change first. [releng]
17:43 <dpifke> Cherry-picking https://gerrit.wikimedia.org/r/c/operations/puppet/+/639216 in beta. [releng]
16:12 <dpifke> Cherry-picking https://gerrit.wikimedia.org/r/c/operations/puppet/+/639197 in beta, should only affect deployment-webperf11. [releng]
2020-11-02 §
22:18 <hashar> Killed Pipeline job , stuck running `docker rmi --force ee88e6bf8898 96f16ff7774f ceb93158475f 47e2530d59a6` T267075 [releng]
2020-10-30 §
20:07 <longma> reloading Zuul to deploy https://gerrit.wikimedia.org/r/c/integration/config/+/631864 [releng]
18:16 <hashar> Successfully tagged docker-registry.discovery.wmnet/releng/ecs:0.0.2-1 # T234565 [releng]
17:48 <hashar> Successfully tagged docker-registry.discovery.wmnet/releng/ecs:0.0.1-1 # T234565 [releng]
2020-10-29 §
17:24 <andrewbogott> signing pending puppet certs for deployment-mediawiki-07.deployment-prep.eqiad1.wikimedia.cloud and deployment-mediawiki-09.deployment-prep.eqiad1.wikimedia.cloud [releng]
17:23 <andrewbogott> signing pending puppet certs for deployment-kafka* nodes [releng]
16:57 <James_F> Zuul: Add CI for node-rdkafka-factory T266058 [releng]
16:17 <andrewbogott> removing jkroll as a project member; the registered email is invalid so probably this user is no longer involved [releng]
14:26 <hashar> restarted Jenkins on releases1002 (java upgrade) [releng]
2020-10-28 §
22:52 <hashar> integration: make sure docker daemon no more has debug enabled by reloading it fleet wide: sudo cumin --force 'name:docker' 'systemctl reload docker' [releng]
19:03 <James_F> Zuul: [mediawiki/extensions/FundraisingEmailUnsubscribe] Skip phan [releng]
18:40 <James_F> Zuul: [mediawiki/extensions/FundraisingEmailUnsubscribe] Use composer quibble [releng]
14:14 <thcipriani> arm keyholder on beta cluster to fix beta-scap-eqiad [releng]
10:00 <hashar> Successfully tagged docker-registry.discovery.wmnet/releng/helm-linter:0.2.10 # T266032 [releng]
2020-10-27 §
22:46 <dpifke> Cherry-picking https://gerrit.wikimedia.org/r/c/operations/puppet/+/636759 in beta puppet. Should only affect deployment-webperf12. [releng]
13:20 <hashar> Enable Docker daemon debug mode on integration-agent-docker-1001 via Horizon https://horizon.wikimedia.org/project/instances/837fff63-ed52-4c2f-823d-936801056fd2/ # T265615 [releng]
13:19 <hashar> Enabling puppet again on CI agent (that also causes Docker/containerd to be rolled back) # T265615 [releng]
2020-10-26 §
22:21 <James_F> Zuul: [mediawiki/extensions/MediaSearch] Add WikibaseCirrus dependency T265939 [releng]
12:00 <hashar> Upgrading docker-ce to 19.03.13 and containerd to 1.3.7 on integration-agent-docker-1001, integration-agent-docker-1002, integration-agent-docker-1003 and integration-agent-docker-1004 # T265615 [releng]
11:12 <hashar> Upgrading containerd.io from 1.2.10 to 1.2.13-2 on integration-agent-docker-1005, integration-agent-docker-1006, integration-agent-docker-1007, integration-agent-docker-1008 # T265615 [releng]
11:03 <hashar> Bring back integration-agent-docker-1020 . It is not the only one affected by T265615 which probably rules out Ceph as the slowness root cause (T260916) [releng]
10:56 <hashar> Upgrading containerd.io from 1.2.10 to 1.2.13-2 on integration-agent-docker-1001, integration-agent-docker-1002, integration-agent-docker-1003, integration-agent-docker-1004 # T265615 [releng]
02:54 <Reedy> Reloading Zuul to deploy https://gerrit.wikimedia.org/r/636229 T266424 [releng]
2020-10-21 §
18:16 <longma> Reloading Zuul to deploy https://gerrit.wikimedia.org/r/c/integration/config/+/635277 [releng]
17:57 <marxarelli> Reloading Zuul to deploy https://gerrit.wikimedia.org/r/c/integration/config/+/635398 [releng]
17:31 <marxarelli> generated 2 new jobs to deploy https://gerrit.wikimedia.org/r/c/integration/config/+/635398 [releng]
2020-10-20 §
22:15 <James_F> Zuul: [mediawiki/extensions/MediaSearch] Install CI for this new prod repo T265939 [releng]
22:14 <James_F> Zuul: [mediawiki/extensions/IPInfo] Disable phan for now [releng]
14:26 <hashar> Switched CI jobs to rust 1.47 https://gerrit.wikimedia.org/r/c/integration/config/+/633693 [releng]