201-250 of 10000 results (48ms)
2018-10-04 ยง
19:15 <ppchelko@deploy1001> Finished deploy [cpjobqueue/deploy@6dc89c0]: Bump cirrusSearchLinksUpdate concurrency to 50 (duration: 00m 53s) [production]
19:14 <ppchelko@deploy1001> Started deploy [cpjobqueue/deploy@6dc89c0]: Bump cirrusSearchLinksUpdate concurrency to 50 [production]
18:49 <sbisson@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:460202|]] (duration: 00m 59s) [production]
18:24 <XioNoX> bounce lvs1002:eth1 switch port [production]
18:23 <sbisson@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:464510|Enable PageTriage/ORES on enwiki (T206149)]] (duration: 01m 01s) [production]
18:21 <bblack> lvs1002: puppet disabled, stopping pybal (fail to 1005) [production]
18:07 <_joe_> disabled notifications for etcd replication lag on conf1005, not in production [production]
17:47 <banyek> repooling labsb1010 (T195747) [production]
17:41 <_joe_> uploaded new python-etcd packages for jessie, stretch [production]
17:38 <XioNoX> asw2-b-eqiad recabling done - T201039 [production]
17:34 <elukey> pool kafka1002 (eventbus) after maintenance [production]
17:22 <elukey> re-enable ircecho after alarms shower [production]
17:15 <andrewbogott> triggering some alerts on labvirt1018 to figure out about alert thresholds [production]
17:06 <elukey> stop ircecho on einstenium - alarms shower [production]
17:02 <gtirloni> tools - published updated toollabs-* Docker images [production]
16:54 <ejegg> updated standalone SmashPig deploy from 82f9d49c23 to 5f21d3f2db [production]
16:52 <XioNoX> Step 3) Add missing links - T201039 [production]
16:45 <shdubsh> etherpad1001 running systemctl reset-failed [production]
16:41 <XioNoX> Connect/enable fpc2:0/51-fpc5:1/0 (5m DAC) - T201039 [production]
16:39 <XioNoX> Enable fpc5-fpc7 - T201039 [production]
16:33 <twentyafterfour> started phd on phab1001 and re-enabled puppet (I had it disabled to prevent starting phd during read-only) [production]
16:25 <twentyafterfour> phabricator is read-write [production]
16:21 <jynus> reloading dbproxy1003,8 [production]
16:16 <marostegui> Stop and reboot db1072 (phabricator master) for maintenance [production]
16:16 <twentyafterfour> phabricator is read-only [production]
16:14 <XioNoX> Enable all VC ports on FPC2 and FPC7 - T201039 [production]
16:13 <XioNoX> starting asw2-b-eqiad re-cabling - T201039 [production]
16:08 <twentyafterfour> logged downtime for phabricator in icinga, stopped phd queue processing in preparation for read-only mode [production]
16:07 <jynus> reloading haproxy @ dbproxy1005 [production]
16:00 <marostegui> Stop MySQL on db1073 for mariadb and kernel upgrade - T201039 T148507 [production]
15:58 <arturo> icinga downtime every server in the main cloudvps deployment for 2h T201039 [production]
15:56 <arturo> icinga downtime every server with the cloudXXXX scheme for 2h T201039 [production]
15:54 <ppchelko@deploy1001> Finished deploy [cpjobqueue/deploy@55dbb8b]: Proper reconnect on topics change T199444 (duration: 00m 55s) [production]
15:53 <ppchelko@deploy1001> Started deploy [cpjobqueue/deploy@55dbb8b]: Proper reconnect on topics change T199444 [production]
15:52 <ppchelko@deploy1001> Finished deploy [changeprop/deploy@5d00448]: Proper reconnect on topics change T199444 (duration: 01m 40s) [production]
15:51 <ppchelko@deploy1001> Started deploy [changeprop/deploy@5d00448]: Proper reconnect on topics change T199444 [production]
15:41 <elukey> depool kafka1002 from eventbus as precautionary step for T201039 [production]
14:48 <banyek> depooling labsb1010 (T195747) [production]
14:09 <marostegui> Sanitize enwikivoyage cebwiki shwiki srwiki mgwiktionary on db1124:3315 T184805 [production]
13:46 <pmiazga@deploy1001> Finished deploy [proton/deploy@ecb9a0e]: Bugfix:handle undefined response and fix grafana stats (T186748,T201158) (duration: 02m 55s) [production]
13:43 <pmiazga@deploy1001> Started deploy [proton/deploy@ecb9a0e]: Bugfix:handle undefined response and fix grafana stats (T186748,T201158) [production]
13:14 <banyek> muting alerts on s2replication @dbstore2002 and resuming compression of s2 database tables (T204930) [production]
13:14 <banyek> muting alerts on dbstore2002 and resuming compression of s2 database tables (T204930) [production]
12:23 <elukey> deploy etcdmirror on conf1005 - T205814 [production]
12:06 <zeljkof> EU SWAT finished [production]
12:06 <zfilipin@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:464481|Add permission "move-rootuserpages" to usergroup "eliminator" at ptwiki (T205595)]] (duration: 00m 57s) [production]
12:01 <moritzm> rolling reboot of ms-fe hosts in codfw for kernel security update [production]
12:00 <zeljkof> one more patch for EU SWAT [production]
11:57 <zeljkof> EU SWAT finished [production]
11:57 <zfilipin@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:460700|Add *.nasimonline.ir to wgCopyUploadsDomains whitelist for Commons (T203371)]] (duration: 00m 56s) [production]