1001-1050 of 10000 results (72ms)
2019-07-24 §
15:56 <bblack> depooling recdns on dns1001 via confctl - T226782 [production]
15:56 <bblack> depooling recdns on dns1001 via confctl [production]
15:47 <jijiki> Rolling puppet-enable and apache reload of jobrunners in eqiad [production]
15:44 <jeh> rebooting labstore1007.wikimedia.org for updates T224228 [production]
15:42 <jijiki> Disable puppet on jobrunners for 525306 [production]
15:11 <herron> resume ingesting [message] =~ /^SlowTimer/ logs on logstash1007 (as a canary) [production]
15:02 <XioNoX> re-enable vc link between asw2-a6 and asw2-a7 - T228823 [production]
14:58 <jeh> unmounting dumps NFS clients from labstore1007.wikimedia.org T224228 [production]
14:54 <XioNoX> cleared vc ports stats on asw2-a-eqiad - T228823 [production]
14:43 <marostegui> Drop abuse_filter_log.afl_log_id in s5 eqiad - T226851 [production]
14:40 <marostegui> Drop abuse_filter_log.afl_log_id in s5 codfw (lag will appear on codfw) - T226851 [production]
14:31 <tarrow@> helmfile [CODFW] Ran 'apply' command on namespace 'termbox' for release 'production' . [production]
13:49 <robh> rebooting cloudvirt1015 into OS, memory error confirmed. new memory replacement dispatch entered via T220853 [production]
13:31 <marostegui> Drop abuse_filter_log.afl_log_id in s2 eqiad - T226851 [production]
13:25 <robh> rebooting cloudvirt1015 into memtest for dell support repair via T220853 [production]
13:06 <liw@deploy1001> Synchronized php: group1 wikis to 1.34.0-wmf.15 (duration: 00m 54s) [production]
13:05 <liw@deploy1001> rebuilt and synchronized wikiversions files: group1 wikis to 1.34.0-wmf.15 [production]
12:19 <marostegui> Stop haproxy on dbproxy1004 and dbproxy1009 (m4 - eventlogging) - T228768 [production]
11:23 <awight@deploy1001> Synchronized wmf-config/CommonSettings.php: [[gerrit:525254|Disable FileImporter source wiki edits (T228851)]] (duration: 00m 54s) [production]
11:12 <awight@deploy1001> Synchronized wmf-config/InitialiseSettings.php: [[gerrit:514672|Remove Content Translation event logging config]] (part 2/2) (duration: 00m 54s) [production]
11:10 <awight@deploy1001> Synchronized wmf-config/CommonSettings.php: [[gerrit:514672|Remove Content Translation event logging config]] (part 1/2) (duration: 00m 59s) [production]
10:04 <marostegui> Drop abuse_filter_log.afl_log_id from labswiki (wikitech) and labtestwiki - T226851 [production]
09:39 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: More traffic to db1082 (duration: 00m 55s) [production]
08:58 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Repool db1082 into API after upgrade (duration: 00m 55s) [production]
08:53 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Slowly repool db1082 after upgrade (duration: 00m 54s) [production]
08:40 <marostegui> Stop MySQL on db1082 for upgrade [production]
08:40 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Depool db1082 for upgrade (duration: 00m 57s) [production]
08:34 <marostegui> Drop abuse_filter_log.afl_log_id in s2 codfw (lag will appear on codfw) - T226851 [production]
07:57 <marostegui> Drop abuse_filter_log.afl_log_id from wikidata in eqiad - T226851 [production]
07:21 <marostegui> Stop MySQL on db1117:3322 to check dbproxy1013 notifications - T202367 [production]
07:10 <marostegui> Deploy grants for dbproxy1013 in m2 - T202367 [production]
05:00 <marostegui> Stop puppet on dbprov2001 to generate s5 mysqldump manually [production]
04:52 <tstarling@deploy1001> Synchronized php-1.34.0-wmf.14/includes/MediaWiki.php: T227700 (duration: 00m 54s) [production]
04:51 <tstarling@deploy1001> Synchronized php-1.34.0-wmf.14/includes/specials/SpecialGoToInterwiki.php: T227700 (duration: 00m 54s) [production]
04:50 <tstarling@deploy1001> Synchronized php-1.34.0-wmf.14/includes/MediaWiki.php: T227700 (duration: 00m 53s) [production]
04:49 <tstarling@deploy1001> Synchronized php-1.34.0-wmf.14/includes/specials/SpecialGoToInterwiki.php: T227700 (duration: 00m 54s) [production]
04:46 <tstarling@deploy1001> Synchronized php-1.34.0-wmf.15/includes/MediaWiki.php: T227700 (duration: 00m 54s) [production]
04:45 <tstarling@deploy1001> Synchronized php-1.34.0-wmf.15/includes/specials/SpecialGoToInterwiki.php: T227700 (duration: 00m 54s) [production]
04:42 <tstarling@deploy1001> Synchronized php-1.34.0-wmf.15/includes/MediaWiki.php: T227700 (duration: 00m 54s) [production]
04:40 <tstarling@deploy1001> Synchronized php-1.34.0-wmf.15/includes/specials/SpecialGoToInterwiki.php: (no justification provided) (duration: 00m 56s) [production]
03:41 <tstarling@deploy1001> Synchronized w/fatal-error.php: Adding post-send exception test for T228462 (duration: 00m 54s) [production]
03:39 <tstarling@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Adding DeferredUpdates log channel (T228462) (duration: 00m 56s) [production]
02:01 <eileen> payments-wiki revision changed from 224c6b2d7b to a28ad541ed, config revision is 8dcb77cf22 [production]
2019-07-23 §
23:44 <eileen> civicrm revision changed from 88e9f24893 to 875ab97742, config revision is 4006d3bdc5 [production]
23:43 <shdubsh> reverting logstash mitigations and re-enable puppet [production]
23:42 <jforrester@deploy1001> Synchronized php-1.34.0-wmf.15/includes/diff/DifferenceEngine.php: T228766 Don't double wrap rollback links (duration: 00m 56s) [production]
23:31 <mutante> mw1267 - rm -rf /srv/mediawiki/php-1.33.0-wmf.23 ; rm -rf /srv/mediawiki/php-1.32.0-wmf.3 ; scap pull [production]
23:25 <dzahn@cumin1001> conftool action : set/pooled=yes; selector: name=mw1267.eqiad.wmnet [production]
22:36 <mutante> rolling out scap 3.11.1-1 on mw-eqiad servers [production]
22:14 <mutante> continuing rollout of new scap version 3.11.1-1, starting with kafka-all followed by other cumin-alias groups (T228328) [production]