951-1000 of 10000 results (71ms)
2019-07-18 ยง
16:36 <jmm@cumin1001> END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) [production]
16:29 <XioNoX> upgrade Routinator to 0.5.0 in eqiad - T220669 [production]
16:24 <krinkle@deploy1001> Synchronized php-1.34.0-wmf.14/resources/src/mediawiki.misc-authed-ooui/special.movePage.js: e97a284dbe54 (duration: 00m 58s) [production]
16:17 <jmm@cumin1001> START - Cookbook sre.ganeti.makevm [production]
16:06 <XioNoX> upgrade Routinator to 0.5.0 in codfw - T220669 [production]
16:05 <XioNoX> add routinator 0.5.0 to APT [production]
15:54 <fsero> depool ms-fe2005 - T228196 [production]
15:40 <hashar@deploy1001> rebuilt and synchronized wikiversions files: Revert group2 wikis to 1.34.0-wmf.13 # T228436 T220739 [production]
15:19 <gehel@cumin2001> END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) [production]
14:46 <godog> roll-restart thumbor in codfw - T228086 [production]
14:45 <gehel@cumin1001> END (FAIL) - Cookbook sre.postgresql.postgres-init (exit_code=99) [production]
14:37 <liw> all wikis at 1.34.0-wmf.14 [production]
14:36 <liw@deploy1001> rebuilt and synchronized wikiversions files: all wikis to 1.34.0-wmf.14 [production]
14:28 <bblack> cp hosts: apt autoremove to clean up pkgs on the fleet [production]
14:27 <nuria@deploy1001> Finished deploy [analytics/refinery@4f07755]: deploying v0.0.94 of refinery (duration: 00m 20s) [production]
14:26 <nuria@deploy1001> Started deploy [analytics/refinery@4f07755]: deploying v0.0.94 of refinery [production]
14:24 <godog> repool thumbor2003 [production]
14:20 <godog> reboot thumbor2003 [production]
14:17 <jijiki> Depool thumbor2003 for reboot [production]
14:12 <gehel@cumin2001> START - Cookbook sre.wdqs.data-transfer [production]
13:53 <moritzm> installing php5 security updates [production]
13:50 <gehel@cumin2001> END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) [production]
13:36 <jeh> rebooting labstore1005.eqiad.wmnet - T224228 [production]
13:34 <jbond42> remove mtail 3.0.0~rc24.1-1+wmf1 from stretch-wikimedia [production]
13:30 <liw@deploy1001> Synchronized php: group1 wikis to 1.34.0-wmf.14 (duration: 00m 53s) [production]
13:29 <liw@deploy1001> rebuilt and synchronized wikiversions files: group1 wikis to 1.34.0-wmf.14 [production]
13:24 <jbond42> downgrade cp servers backl to 3.0.0~rc5-1~bpo9+1 [production]
13:23 <liw> promoting 1.34.0-wmf.14 to group1 [production]
13:22 <godog> temporarily stop ircecho on icinga1001 to avoid spam [production]
13:00 <jbond42> rolling upgrade of mtail [production]
12:57 <gehel@cumin2001> START - Cookbook sre.wdqs.data-transfer [production]
12:53 <gehel@cumin1001> START - Cookbook sre.wdqs.data-transfer [production]
12:53 <gehel@cumin1001> END (ERROR) - Cookbook sre.wdqs.data-transfer (exit_code=97) [production]
12:51 <gehel@cumin1001> START - Cookbook sre.wdqs.data-transfer [production]
12:34 <gehel@cumin1001> START - Cookbook sre.postgresql.postgres-init [production]
12:26 <jbond42> add mtail 3.0.0~rc24.1-1+wmf1 to stretch-wikimedia [production]
11:13 <dcausse> EU Swat done [production]
11:08 <dcausse@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Revert [cirrus] switch search traffic (except completion) to codfw (duration: 00m 56s) [production]
11:02 <godog> swift eqiad-prod: put back ms-be1043 sdk1 - T218544 [production]
10:51 <gehel@cumin1001> END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) [production]
10:43 <ema> cp-eqiad: varnish frontend rolling restarts for 5.1.3-1wm11 upgrades T227672 [production]
10:37 <jijiki> enable puppet on services_proxy hosts - T228063 [production]
10:29 <godog> reboot wezen.codfw.wmnet - T225713 [production]
10:27 <gehel@cumin1001> END (FAIL) - Cookbook sre.postgresql.postgres-init (exit_code=99) [production]
10:15 <jijiki> Disable puppet on services_proxy hosts - T228063 [production]
09:33 <gehel@cumin1001> START - Cookbook sre.wdqs.data-transfer [production]
09:26 <gehel@cumin1001> END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) [production]
09:09 <godog> resume swift ms-be rolling restarts - T225713 [production]
09:03 <fsero> reuploding missing layers T228196 [production]
08:57 <hashar> contint1001: stopped zuul, ran apt install to get the new python2.7 copied to Zuul virtualenv, restarted zuul/zuul-merger. That clears a couple Icinga alarms from yesterday [production]