651-700 of 10000 results (50ms)
2018-11-26 §
12:00 <kartik@deploy1001> Finished deploy [cxserver/deploy@da41227]: Disable Youdao MT until service is back (duration: 04m 01s) [production]
11:56 <kartik@deploy1001> Started deploy [cxserver/deploy@da41227]: Disable Youdao MT until service is back [production]
11:41 <arturo> icinga downtime cloudcontrol1004 for some systemd slice tests [production]
11:01 <jdrewniak@deploy1001> Synchronized portals: Wikimedia Portals Update: [[gerrit:475721|Bumping portals to master (T128546)]] (duration: 00m 46s) [production]
11:00 <jdrewniak@deploy1001> Synchronized portals/wikipedia.org/assets: Wikimedia Portals Update: [[gerrit:475721|Bumping portals to master (T128546)]] (duration: 00m 47s) [production]
10:32 <moritzm> rebooting boron [production]
09:58 <banyek> performing schema change on s6 master - db1061 for frwiki (T85757) [production]
09:51 <banyek> performing schema change on s6 master - db1061 for ruwiki (T85757) [production]
09:44 <banyek> performing schema change on s6 master - db1061 for jawiki (T85757) [production]
08:35 <moritzm> removed labvirt1015 from debmonitor DB (got renamed to cloudvirt1015) [production]
08:22 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Repool db1096:3316 - T202167 (duration: 00m 46s) [production]
08:16 <marostegui> Deploy schema change on db1096:3316 - T202167 [production]
08:16 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Depool db1096:3316 - T202167 (duration: 00m 47s) [production]
08:10 <moritzm> installing pixman security updates [production]
07:36 <elukey> restart memcached on mc1021 (cache wipe) to add -R 200 - T208844 [production]
07:10 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Repool db1088 - T86338 (duration: 00m 46s) [production]
07:00 <marostegui> Deploy schema change on db1088 - T86338 [production]
06:59 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Depool db1088 - T86338 (duration: 00m 46s) [production]
06:46 <marostegui> Deploy schema change on db2067 - T86338 [production]
06:36 <marostegui> Deploy schema change on s3 master (db1075) - T86339 [production]
06:28 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Repool db1077 - T86339 (duration: 00m 46s) [production]
06:22 <marostegui> Reload haproxy on dbproxy1005 [production]
06:21 <marostegui> Deploy schema change on db1077 (s3 sanitarium master) with replication - T86339 [production]
06:21 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Depool db1077 - T86339 (duration: 00m 50s) [production]
06:02 <kartik@deploy1001> Finished deploy [cxserver/deploy@d2173ca]: Update cxserver to 218a543 (duration: 04m 21s) [production]
05:58 <kartik@deploy1001> Started deploy [cxserver/deploy@d2173ca]: Update cxserver to 218a543 [production]
2018-11-25 §
10:40 <_joe_> restarting pdfrender on scb1002, flapping since 3:00 UTC [production]
2018-11-23 §
21:15 <bawolff> deploy patch for T210192 [production]
16:53 <bblack> cleaned up remnants of globalsign-2017 unified cert (OCSP cache/config, unmanaged cert files, etc) on all cpNNNN - T206804 [production]
14:02 <gehel> restor wdqs-updater heap to 2G - T210235 [production]
13:32 <moritzm> installing confuse security updates [production]
12:04 <gehel> manually increasing wdqs-updater heap to 4G - T210235 [production]
11:37 <gehel> restarting updater on all wdqs ndoes [production]
08:41 <akosiaris@puppetmaster1001> conftool action : set/pooled=yes; selector: dc=.*,service=zotero,cluster=kubernetes,name=.* [production]
08:10 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Repool db1078 - T86339 (duration: 00m 46s) [production]
07:48 <moritzm> installing libtirpc security updates [production]
07:14 <marostegui> Deploy schema change db1078 - T86339 [production]
07:14 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Depool db1078 - T86339 (duration: 00m 45s) [production]
07:00 <marostegui> Deploy schema change db1095 - T86339 [production]
06:59 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Repool db1123 - T86339 (duration: 00m 49s) [production]
06:32 <marostegui> Deploy schema change db1123 - T86339 [production]
06:31 <marostegui> Deploy schema change dbstore1002:s3 - T86339 [production]
06:29 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Depool db1123 - T86339 (duration: 00m 48s) [production]
06:15 <marostegui> Deploy schema change on s3 codfw master (db2043) with replication - T86339 [production]
06:13 <marostegui> Deploy schema change on db1067 (s1) master - T86339 [production]
2018-11-22 §
21:08 <godog> disable raid handler for ms-be2021 - T208096 [production]
19:01 <moritzm> installing uriparser security updates [production]
18:46 <moritzm> installing openjpeg2 security updates [production]
18:45 <arturo> enable puppet in all CloudVPS HW servers [production]
18:37 <arturo> disable puppet in all CloudVPS HW servers to test a patch (T209948) [production]