8451-8500 of 10000 results (40ms)
2020-07-05 §
21:46 <qchris> Restarting gerrit on gerrit2001 to pick up new war and jars. [production]
21:45 <qchris@deploy1001> Finished deploy [gerrit/gerrit@fbd0684]: Bump gerrit to 3.2.2-102-g3bbb138e13, zuul plugin to master-0-g7accc67, and gitiles to v3.2.2-1-g00c5ca0-with-0e3b533 on gerrit2001 (duration: 00m 10s) [production]
21:45 <qchris@deploy1001> Started deploy [gerrit/gerrit@fbd0684]: Bump gerrit to 3.2.2-102-g3bbb138e13, zuul plugin to master-0-g7accc67, and gitiles to v3.2.2-1-g00c5ca0-with-0e3b533 on gerrit2001 [production]
21:32 <qchris> Restarting gerrit on gerrit1002 to pick up new wars and jars. [production]
21:32 <qchris@deploy1001> Finished deploy [gerrit/gerrit@fbd0684]: Bump gerrit to 3.2.2-102-g3bbb138e13 and zuul plugin to master-0-g7accc67 (duration: 00m 08s) [production]
21:32 <qchris@deploy1001> Started deploy [gerrit/gerrit@fbd0684]: Bump gerrit to 3.2.2-102-g3bbb138e13 and zuul plugin to master-0-g7accc67 [production]
21:20 <qchris> Enable puppet on gerrit1002 (gerrit-test) again to let it catch up again [production]
16:01 <gehel> restart elastic-psi on elastic1052 (high GC rate) [production]
15:56 <gehel> restart blazegraph + updater on wdqs1007 and depool to allow catching up on lag [production]
2020-07-04 §
19:23 <qchris@deploy1001> Finished deploy [gerrit/gerrit@b78914b]: Bump gitiles to v3.2.2-1-g00c5ca0-with-0e3b533 on gerrit1002 (duration: 00m 08s) [production]
19:23 <qchris@deploy1001> Started deploy [gerrit/gerrit@b78914b]: Bump gitiles to v3.2.2-1-g00c5ca0-with-0e3b533 on gerrit1002 [production]
14:05 <qchris> Disable puppet on gerrit1002 (gerrit-test) to deploy Gerrit UI updates there to gather feedback [production]
12:42 <reedy@deploy1001> Synchronized wmf-config/interwiki.php: Update interwiki cache (duration: 02m 24s) [production]
02:28 <reedy@deploy1001> Synchronized php-1.35.0-wmf.39/extensions/Score/includes/Score.php: Short circuit lilypond version check to allow usage of cached files T257066 (duration: 00m 55s) [production]
2020-07-03 §
21:49 <reedy@deploy1001> Synchronized php-1.35.0-wmf.39/extensions/Score/: Sync maintenance script (duration: 00m 58s) [production]
18:47 <cdanis> ✔️ cdanis@an-coord1001.eqiad.wmnet ~ 🕒☕ sudo systemctl restart hive-server2.service [production]
16:51 <krinkle@deploy1001> Synchronized wmf-config/CommonSettings.php: Ifa929b2ad4 (duration: 00m 57s) [production]
16:02 <reedy@deploy1001> Synchronized wmf-config/CommonSettings.php: Rename wgRestrictionMethod to wgShellRestrictionMethod (duration: 00m 58s) [production]
15:46 <jayme@cumin1001> END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) [production]
15:43 <jayme@cumin1001> END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) [production]
15:43 <jynus@cumin1001> dbctl commit (dc=all): 'Reduce db1118 weight to spread load mode evenly', diff saved to https://phabricator.wikimedia.org/P11730 and previous config saved to /var/cache/conftool/dbconfig/20200703-154337-jynus.json [production]
15:40 <jayme@cumin1001> START - Cookbook sre.ganeti.makevm [production]
15:38 <jayme@cumin1001> START - Cookbook sre.ganeti.makevm [production]
15:09 <elukey@cumin1001> END (PASS) - Cookbook sre.hadoop.stop-cluster (exit_code=0) [production]
15:02 <elukey@cumin1001> START - Cookbook sre.hadoop.stop-cluster [production]
14:11 <elukey@cumin1001> END (FAIL) - Cookbook sre.hadoop.stop-cluster (exit_code=99) [production]
14:11 <_joe_> restarted php-fpm on wtp1033, stuck in sigill [production]
13:59 <elukey@cumin1001> START - Cookbook sre.hadoop.stop-cluster [production]
12:41 <hashar> Restarting Zuul / CI [production]
11:39 <jmm@cumin2001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) [production]
11:36 <jmm@cumin2001> START - Cookbook sre.hosts.reboot-single [production]
11:32 <jmm@cumin2001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) [production]
11:29 <jmm@cumin2001> START - Cookbook sre.hosts.reboot-single [production]
11:29 <moritzm> rebooting urldownloader standby hosts for kernel updates (1002/2002) [production]
10:59 <moritzm> installing json-c security updates on jessie [production]
10:51 <moritzm> installing ruby-json security updates [production]
10:25 <moritzm> installing nss security updates on jessie [production]
10:18 <jmm@cumin2001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) [production]
10:15 <elukey> notebook1004 renamed to an-scheduler1001 [production]
10:15 <jmm@cumin2001> START - Cookbook sre.hosts.reboot-single [production]
10:09 <elukey@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
10:07 <elukey@cumin1001> START - Cookbook sre.hosts.downtime [production]
09:06 <jmm@cumin2001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) [production]
09:04 <jmm@cumin2001> START - Cookbook sre.hosts.reboot-single [production]
09:00 <jmm@cumin2001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) [production]
08:58 <jmm@cumin2001> START - Cookbook sre.hosts.reboot-single [production]
08:58 <jmm@cumin2001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) [production]
08:56 <jmm@cumin2001> START - Cookbook sre.hosts.reboot-single [production]
08:55 <jmm@cumin2001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) [production]
08:51 <jmm@cumin2001> START - Cookbook sre.hosts.reboot-single [production]