1751-1800 of 10000 results (63ms)
2019-07-05 ยง
15:20 <jmm@cumin2001> START - Cookbook sre.ganeti.makevm [production]
15:15 <jmm@cumin2001> END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) [production]
15:05 <jmm@cumin2001> START - Cookbook sre.ganeti.makevm [production]
15:01 <jmm@cumin2001> END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) [production]
14:51 <jmm@cumin2001> START - Cookbook sre.ganeti.makevm [production]
14:24 <jmm@cumin2001> END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) [production]
14:15 <jmm@cumin2001> START - Cookbook sre.ganeti.makevm [production]
13:44 <elukey> roll restart of aqs on aqs100* to pick up new druid settings [production]
13:33 <fsero> disabling puppet on swift backends [production]
13:26 <fsero> restarting swift-container-sync on swift backends [production]
13:05 <ema> pool cp1090 w/ ATS backend T226638 [production]
12:12 <ema> depool cp1090 and reimage as upload_ats T226638 [production]
11:46 <ema> pool cp1088 w/ ATS backend T226638 [production]
11:39 <jiji@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
11:39 <jiji@cumin1001> START - Cookbook sre.hosts.downtime [production]
11:38 <jijiki> Reboot ms-be1021 - T141756 - T227076 [production]
11:32 <jijiki> Upgrading smartarray firmware on ms-be1021 - T141756 - T227076 [production]
11:31 <moritzm> installing postgresql-9.4 updates on jessie [production]
11:10 <jmm@cumin1001> END (FAIL) - Cookbook sre.ganeti.makevm (exit_code=99) [production]
11:09 <jmm@cumin1001> START - Cookbook sre.ganeti.makevm [production]
11:05 <jmm@cumin2001> END (FAIL) - Cookbook sre.ganeti.makevm (exit_code=99) [production]
11:05 <jmm@cumin2001> START - Cookbook sre.ganeti.makevm [production]
11:05 <jmm@cumin2001> END (FAIL) - Cookbook sre.ganeti.makevm (exit_code=99) [production]
11:04 <jmm@cumin2001> START - Cookbook sre.ganeti.makevm [production]
11:00 <ema> depool cp1088 and reimage as upload_ats T226638 [production]
10:55 <ema> pool cp1086 w/ ATS backend T226638 [production]
10:29 <moritzm> rebooting debug proxies to pick up MDS-enabled qemu [production]
10:29 <jmm@cumin2001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
10:29 <jmm@cumin2001> START - Cookbook sre.hosts.downtime [production]
10:23 <jiji@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
10:23 <jiji@cumin1001> START - Cookbook sre.hosts.downtime [production]
10:23 <moritzm> rebooting seaborgium to pick up correct Stretch kernel [production]
10:15 <moritzm> rebooting serpens to pick up correct Stretch kernel [production]
10:14 <moritzm> fixed up kernel packages on serpens/seaborgium, these were dist-upgraded from jessie, but the correct kernel packages for Stretch were not setup, as such there were still stuck with an old jessie kernel [production]
10:06 <jiji@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
10:06 <jiji@cumin1001> START - Cookbook sre.hosts.downtime [production]
10:02 <jijiki> Rolling rebood rdb* hosts - T227304 [production]
10:00 <moritzm> rebooting seaborgium to pick up MDS-enabled qemu [production]
09:51 <moritzm> rebooting serpens to pick up MDS-enabled qemu [production]
09:48 <jmm@cumin2001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
09:48 <jmm@cumin2001> START - Cookbook sre.hosts.downtime [production]
09:39 <ema> depool cp1086 and reimage as upload_ats T226638 [production]
09:31 <moritzm> rebooting LDAP replicas in eqiad [production]
09:28 <jmm@cumin2001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
09:28 <jmm@cumin2001> START - Cookbook sre.hosts.downtime [production]
09:15 <gehel@puppetmaster1001> conftool action : set/pooled=inactive; selector: dc=codfw,name=elastic2054.codfw.wmnet [production]
09:01 <moritzm> rebooting kraz (irc.wikimedia.org) to pick up MDS-enabled qemu [production]
08:54 <jmm@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
08:54 <jmm@cumin1001> START - Cookbook sre.hosts.downtime [production]
07:57 <jynus@deploy1001> Synchronized wmf-config/db-eqiad.php: Repool db1087 (duration: 00m 48s) [production]