6351-6400 of 10000 results (71ms)
2018-11-22 §
09:06 <moritzm> installing jasper security updates [production]
08:21 <marostegui> Deploy schema change on s1 codfw master (db2048) with replication - T86339 [production]
08:19 <marostegui> Deploy schema change on db1062 (s7 master) - T86339 [production]
08:19 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Repool db1094 - T86339 (duration: 00m 46s) [production]
08:17 <marostegui> Deploy schema change on db1094 - T86339 [production]
08:16 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Depool db1094 - T86339 (duration: 00m 49s) [production]
06:58 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Repool db1086 - T86339 (duration: 00m 46s) [production]
06:55 <marostegui> Deploy schema change on db1086 - T86339 [production]
06:55 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Depool db1086 - T86339 (duration: 00m 46s) [production]
06:51 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Repool db1079 - T86339 (duration: 00m 46s) [production]
06:48 <marostegui> Deploy schema change on db1079 (sanitarium master) with replication - T86339 [production]
06:48 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Depool db1079 - T86339 (duration: 00m 51s) [production]
02:25 <mutante> stat1005 - started nagios-nrpe-server [production]
2018-11-21 §
23:34 <mutante> rsyncing /home from rutherfordium.eqiad to people1001.eqiad (people.wikimedia.org) T210036 [production]
21:16 <robh> cp5001 is offline running hardware tests after firmware updates to see if memory error still exists. ref: T199675 [production]
20:55 <robh> cp5001 reboot for firmware update [production]
19:54 <milimetric@deploy1001> Finished deploy [analytics/aqs/deploy@e114d99]: Fixing sorting bug on top endpoints (duration: 05m 34s) [production]
19:49 <milimetric@deploy1001> Started deploy [analytics/aqs/deploy@e114d99]: Fixing sorting bug on top endpoints [production]
19:43 <ejegg> updated fundraising internal dashboard from b01458b260 to 5e9fb9a3ef [production]
17:21 <elukey> manually started systemd-journald.service on scb1001 after OOM [production]
17:20 <jynus> stop and upgrade db2081 [production]
17:04 <jynus> stop and upgrade db2080 [production]
16:40 <jynus> stop and upgrade db2066 [production]
16:37 <bawolff> deploy patch T209794 [production]
16:25 <jynus> stop and upgrade db2063 [production]
15:15 <jynus> stop and upgrade db2073 [production]
15:15 <banyek@deploy1001> Synchronized wmf-config/db-eqiad.php: T85757: repool db1085 (duration: 00m 46s) [production]
15:03 <banyek> repooling db1085 after schema change (T85757) [production]
15:00 <banyek> restarting replication on db1085 (T85757) [production]
14:31 <banyek> stopping replication on db1085 (T85757) [production]
14:27 <banyek@deploy1001> Synchronized wmf-config/db-eqiad.php: T85757: depool db1085 (duration: 00m 46s) [production]
14:22 <banyek> depooling db1085 due schema change (T85757) [production]
13:39 <jynus> stop and upgrade db2077 [production]
13:05 <XioNoX> remove BGP session to 2603 on cr4-ulsfo [production]
12:13 <jynus> stop and upgrade db2076 [production]
12:08 <banyek> running schema change on dbstore1001:3316 (T85757) [production]
12:08 <banyek> running schema change on dbstore1001 (T85757) [production]
12:04 <jynus> stop and upgrade db2075 [production]
11:53 <banyek> running schema change on dbstore1002 (T85757) [production]
11:00 <akosiaris> disable puppet on ores2* ores1* for gradual rollout of https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/474694/1/modules/ores/manifests/web.pp [production]
10:55 <jynus> stop and upgrade db2074 [production]
10:51 <_joe_> uploading prometheus-php-fpm-exporter to stretch-wikimedia main, T209573 [production]
10:44 <banyek@deploy1001> Synchronized wmf-config/db-eqiad.php: T85757: repool db1096:3316 (duration: 00m 46s) [production]
10:42 <banyek> repooling db1096:3316 after schema change (T85757) [production]
10:30 <jynus> stop and upgrade db2095 [production]
10:26 <banyek@deploy1001> Synchronized wmf-config/db-eqiad.php: T85757: depool db1096:3316 (duration: 00m 45s) [production]
10:26 <godog> initial weight for new ms-be2* hosts (all but ms-be2047) - T209395 [production]
10:23 <banyek> depooling db1096:3316 due schema change (T85757) [production]
10:14 <banyek@deploy1001> Synchronized wmf-config/db-eqiad.php: T85757: repool db1098:3316 (duration: 00m 46s) [production]
10:13 <banyek@deploy1001> sync-file aborted: T85757: depool db1098:3316 (duration: 00m 03s) [production]