601-650 of 10000 results (60ms)
2019-07-24 ยง
20:45 <ppchelko@deploy1001> Started deploy [restbase/deploy@7911f65]: Store PCS endpoints T222384 [production]
20:39 <bsitzmann@deploy1001> Finished deploy [mobileapps/deploy@2e2ce6c]: Update mobileapps to 1751a2e (duration: 04m 20s) [production]
20:38 <ppchelko@deploy1001> Finished deploy [changeprop/deploy@bf28187]: Rerender PCS endpoints T222384 (duration: 01m 34s) [production]
20:36 <ppchelko@deploy1001> Started deploy [changeprop/deploy@bf28187]: Rerender PCS endpoints T222384 [production]
20:35 <bsitzmann@deploy1001> Started deploy [mobileapps/deploy@2e2ce6c]: Update mobileapps to 1751a2e [production]
20:12 <jeh> redirecting dumps.wikimedia.org back to labstore1007.wikimedia.org T224228 [production]
19:43 <ejegg> updated fundraising CiviCRM from 875ab97742 to 121feb5d53 [production]
19:08 <legoktm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Enable SecureLinkFixer on group0 wikis - T200751 (duration: 00m 55s) [production]
18:33 <cmjohnson1> moving cloudvirt107 to 10G rack T228691 [production]
18:19 <krinkle@deploy1001> Synchronized php-1.34.0-wmf.15/includes/cache/localisation/LocalisationCache.php: 31d99eb381bc (duration: 00m 54s) [production]
18:15 <ejegg> updated payments-wiki from a28ad541ed to 70b432d309 [production]
18:13 <urandom> creating new restbase keyspaces -- T228804 [production]
18:12 <Krinkle> krinkle@deploy1001: extensions/CheckUser is dirty in php-1.34.0-wmf.15 [production]
17:14 <XioNoX> rollback failover master VIP of ae2.1202 inet6 away from cr1-eqiad - T226782 [production]
17:10 <XioNoX> Add mr1-codfw<->cr1/2-codfw vlan/link config on asw-a-codfw - T228112 [production]
16:44 <jijiki> Rolling puppet-enable and apache reload of jobrunners in codfw [production]
16:12 <bblack@cumin1001> conftool action : set/pooled=yes; selector: name=dns1001.wikimedia.org [production]
16:12 <bblack> re-pooling recdns on dns1001 via confctl - T226782 [production]
16:11 <bblack> lvs1014 - restore puppet and resolv.conf contents, restart pybal [production]
16:10 <bblack> dns1001 - restart recursor and re-enable puppet - T226782 [production]
16:07 <jforrester@deploy1001> Synchronized php-1.34.0-wmf.15/includes/export/XmlDumpWriter.php: T228720 make XmlDumpwriter more resilient to blob store corruption (duration: 00m 55s) [production]
16:06 <jforrester@deploy1001> Synchronized php-1.34.0-wmf.14/includes/export/XmlDumpWriter.php: T228720 make XmlDumpwriter more resilient to blob store corruption (duration: 00m 55s) [production]
15:59 <bblack> dns1001 - puppet disable, stop recursor service to kill anycast advert - T226782 [production]
15:59 <bblack> lvs1014 - puppet disable, remove dns1001 from resolv.conf, restart pybal - T226782 [production]
15:58 <XioNoX> failover master VIP of ae2.1202 inet6 away from cr1-eqiad - T226782 [production]
15:56 <bblack@cumin1001> conftool action : set/pooled=no; selector: name=dns1001.wikimedia.org [production]
15:56 <bblack> depooling recdns on dns1001 via confctl - T226782 [production]
15:56 <bblack> depooling recdns on dns1001 via confctl [production]
15:47 <jijiki> Rolling puppet-enable and apache reload of jobrunners in eqiad [production]
15:44 <jeh> rebooting labstore1007.wikimedia.org for updates T224228 [production]
15:42 <jijiki> Disable puppet on jobrunners for 525306 [production]
15:11 <herron> resume ingesting [message] =~ /^SlowTimer/ logs on logstash1007 (as a canary) [production]
15:02 <XioNoX> re-enable vc link between asw2-a6 and asw2-a7 - T228823 [production]
14:58 <jeh> unmounting dumps NFS clients from labstore1007.wikimedia.org T224228 [production]
14:54 <XioNoX> cleared vc ports stats on asw2-a-eqiad - T228823 [production]
14:43 <marostegui> Drop abuse_filter_log.afl_log_id in s5 eqiad - T226851 [production]
14:40 <marostegui> Drop abuse_filter_log.afl_log_id in s5 codfw (lag will appear on codfw) - T226851 [production]
14:31 <tarrow@> helmfile [CODFW] Ran 'apply' command on namespace 'termbox' for release 'production' . [production]
13:49 <robh> rebooting cloudvirt1015 into OS, memory error confirmed. new memory replacement dispatch entered via T220853 [production]
13:31 <marostegui> Drop abuse_filter_log.afl_log_id in s2 eqiad - T226851 [production]
13:25 <robh> rebooting cloudvirt1015 into memtest for dell support repair via T220853 [production]
13:06 <liw@deploy1001> Synchronized php: group1 wikis to 1.34.0-wmf.15 (duration: 00m 54s) [production]
13:05 <liw@deploy1001> rebuilt and synchronized wikiversions files: group1 wikis to 1.34.0-wmf.15 [production]
12:19 <marostegui> Stop haproxy on dbproxy1004 and dbproxy1009 (m4 - eventlogging) - T228768 [production]
11:23 <awight@deploy1001> Synchronized wmf-config/CommonSettings.php: [[gerrit:525254|Disable FileImporter source wiki edits (T228851)]] (duration: 00m 54s) [production]
11:12 <awight@deploy1001> Synchronized wmf-config/InitialiseSettings.php: [[gerrit:514672|Remove Content Translation event logging config]] (part 2/2) (duration: 00m 54s) [production]
11:10 <awight@deploy1001> Synchronized wmf-config/CommonSettings.php: [[gerrit:514672|Remove Content Translation event logging config]] (part 1/2) (duration: 00m 59s) [production]
10:04 <marostegui> Drop abuse_filter_log.afl_log_id from labswiki (wikitech) and labtestwiki - T226851 [production]
09:39 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: More traffic to db1082 (duration: 00m 55s) [production]
08:58 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Repool db1082 into API after upgrade (duration: 00m 55s) [production]