1401-1450 of 10000 results (54ms)
2019-01-03 §
12:41 <arturo> T212302 reimaging again cloudvirt1030 to test final puppet code [production]
12:33 <banyek@deploy1001> Synchronized wmf-config/db-eqiad.php: depool db1094 for schema change - T85757 (duration: 00m 46s) [production]
12:28 <banyek> depooling db1094 for schema change - T85757 [production]
12:27 <moritzm> restarting tor on torrelay1001 to pick up OpenSSL security update [production]
11:02 <_joe_> manually reloading icinga to pick up changes to commands.cfg [production]
10:55 <moritzm> installing apache updates on puppetmasters [production]
10:22 <moritzm> installing ghostscript security updates on jessie [production]
09:51 <elukey> restart memcached on mc1023 to apply -R 200 - T208844 [production]
09:46 <moritzm> remove imagemagick remnants from ATS hosts (obsoleted by upstream packaging change which dropped the webp plugin) [production]
09:39 <moritzm> installing nginx updates on puppetdb* [production]
09:26 <banyek@deploy1001> Synchronized wmf-config/db-codfw.php: repool es2019 - T212833 (duration: 01m 33s) [production]
09:18 <banyek> repooling es2019 - T212833 [production]
08:46 <moritzm> rolling restart of proton to pick up OpenSSL update [production]
08:35 <banyek> depooled es2019 as host was unsresponsive - T212833 [production]
08:35 <banyek@deploy1001> Synchronized wmf-config/db-codfw.php: depool es2019, host is unsresponsible - T212833 (duration: 00m 49s) [production]
08:11 <moritzm> installing OpenSSL security updates [production]
00:21 <mutante> notebook1004 - started nagios-nrpe-server one more time [production]
2019-01-02 §
23:59 <mutante> notebook1004 still keeps running out of memory from some user actions and that kills nagios-nrpe-server and that causes a bunch of Icinga alerts [production]
23:39 <mutante> notebook1004 - systemctl start nagios-nrpe-server [production]
23:39 <mutante> notebook1004 - systemctl status nagios-nrpe-server [production]
20:59 <herron@puppetmaster1001> conftool action : set/pooled=yes; selector: dc=eqiad,cluster=parsoid,service=parsoid,name=wtp1028.eqiad.wmnet [production]
20:59 <herron> repooling wtp1028 T212624 [production]
20:52 <herron> rebooting wtp1028 — looking for POST errors T212624 [production]
20:05 <Krinkle> mwmaint1002: foreachwikiindblist s5 deleteEqualMessages.php [production]
20:04 <Krinkle> mwmaint1002: foreachwikiindblist s2 deleteEqualMessages.php [production]
18:35 <volans> restarting icinga on icinga1001 T212669 [production]
16:50 <XioNoX> create BGP sessions to AS3214 in AMS-IX [production]
16:46 <XioNoX> remove BGP sessions to AS42949 in AMS-IX (leaving the IX) [production]
16:43 <XioNoX> remove BGP sessions to AS6866 in AMS-IX (leaving the IX) [production]
16:33 <banyek@deploy1001> Synchronized wmf-config/db-eqiad.php: repool db1090:3317 after schema change - T85757 (duration: 00m 46s) [production]
16:30 <arturo> reimaging cloudvirt1030 with stretch, server cleanup after puppet refactoring [production]
16:29 <moritzm> restarting Superset to pick up openssl security update [production]
16:25 <moritzm> restarting Hue to pick up openssl security update [production]
16:23 <arturo> T212302 re-enable puppet in all {cloud,lab}virt* servers, all was fine [production]
16:22 <banyek> repooling db1090:3317 after schema change (T85757) [production]
16:11 <arturo> T212302 disable puppet in all {cloud,lab}virt* servers to merge https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/481194/ [production]
15:39 <banyek@deploy1001> Synchronized wmf-config/db-eqiad.php: depool db1090:3317 for schema change - T85757 (duration: 00m 44s) [production]
15:34 <moritzm> installing OpenSSL security updates [production]
15:31 <banyek> depooling db1090:3317 for schema change (T85757) [production]
15:13 <banyek@deploy1001> Synchronized wmf-config/db-eqiad.php: repool db1086 after schema change - T85757 (duration: 00m 44s) [production]
15:07 <banyek> repooling db1086 after schema change (T85757) [production]
14:49 <banyek> executing schema change on db1086 - T85757 [production]
14:48 <moritzm> installing ghostscript security update for jessie [production]
14:47 <banyek@deploy1001> Synchronized wmf-config/db-eqiad.php: depool db1086 for schema change - T85757 (duration: 00m 45s) [production]
14:38 <banyek> depooling db1086 for schema change (T85757) [production]
14:15 <ema> cp hosts: upgrade OpenSSL from 1.1.0f to 1.1.0j [production]
13:39 <moritzm> installing ghostscript update for stretch [production]
13:33 <moritzm> installing libav security updates [production]
13:30 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Repool db1119 T86338 T202167 (duration: 00m 44s) [production]
13:17 <moritzm> installing openjpeg2 security updates [production]