2151-2200 of 10000 results (72ms)
2018-10-25 §
19:26 <twentyafterfour@deploy1001> Started scap: full sync to be sure that 1.33.0-wmf.1 is fully deployed [production]
19:23 <twentyafterfour> beginning mediawiki train. Will start with group1 and then monitor the situation for a few minutes. If everything looks good then we go to group2. [production]
19:20 <sbisson@deploy1001> Synchronized php-1.32.0-wmf.26/extensions/ContentTranslation/: SWAT: [[gerrit:469603|Add detailed logging for AbuseFilter]] (duration: 00m 56s) [production]
18:37 <sbisson@deploy1001> Synchronized php-1.33.0-wmf.1/extensions/ContentTranslation/: SWAT: [[gerrit:469605|Remove the session parameter from AbuseFilter logging]] (duration: 00m 56s) [production]
17:58 <aaron@deploy1001> Synchronized php-1.33.0-wmf.1/extensions/Translate/tag: c5fa239917a870240ec4dcd8a617f0f8033aa9bf (duration: 00m 55s) [production]
17:52 <aaron@deploy1001> Synchronized php-1.33.0-wmf.1/includes/page/WikiPage.php: f3b5a1df116f426c2809f2a266b9d761f15c349f (duration: 00m 54s) [production]
17:52 <smalyshev@deploy1001> Finished deploy [wdqs/wdqs@4967dba]: Test deploy new update & scripts (duration: 00m 28s) [production]
17:51 <smalyshev@deploy1001> Started deploy [wdqs/wdqs@4967dba]: Test deploy new update & scripts [production]
17:50 <aaron@deploy1001> Synchronized php-1.33.0-wmf.1/tests/phpunit/includes/page/WikiPageDbTestBase.php: f3b5a1df116f426c2809f2a266b9d761f15c349f (duration: 00m 55s) [production]
17:36 <aaron@deploy1001> Synchronized php-1.33.0-wmf.1/includes/changetags/ChangeTags.php: 08f8e6a9d7f1dcb281321c5e3a3471169e68348d (duration: 00m 55s) [production]
17:24 <bsitzmann@deploy1001> Finished deploy [mobileapps/deploy@95452cf]: Update mobileapps to 58cbdff (T206527) (duration: 03m 50s) [production]
17:20 <bsitzmann@deploy1001> Started deploy [mobileapps/deploy@95452cf]: Update mobileapps to 58cbdff (T206527) [production]
17:20 <mutante> planet - regenerating feeds for 'en' and 'de', others will follow by cron. switching to new theme. replaced bootstrap with bulma. removed jQuery. thanks to paladox [production]
16:34 <gehel@puppetmaster1001> conftool action : set/weight=20; selector: dc=eqiad,cluster=wdqs,name=wdqs1005.codfw.wmnet [production]
16:34 <gehel@puppetmaster1001> conftool action : set/weight=20; selector: dc=eqiad,cluster=wdqs,name=wdqs1004.codfw.wmnet [production]
16:34 <gehel> decreasing relative weight of wdqs1003 in LVS to ease the updater [production]
16:24 <shdubsh> installed patched nagios-nrpe-plugin and nagios-nrpe-server on icinga1001 - T207775 [production]
15:36 <elukey> shutdown aqs1006 to replace one broken disk - T206915 [production]
15:31 <SMalyshev> depooling wdqs1003 again, it's not catching up like the other hosts [production]
15:16 <addshore@deploy1001> Synchronized wmf-config/InitialiseSettings-labs.php: [Beta Cluster] Re-enable WBMI on Beta Commons (duration: 00m 54s) [production]
15:11 <addshore@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Revert "logging: Disable Wikibase.NewItemIdFormatter channel" (duration: 00m 55s) [production]
15:08 <addshore@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Explicitly set wgLexemeEnableRepo for wikidatas [[gerrit:469625]] (duration: 00m 55s) [production]
15:02 <godog> test rsyslog 8.38 upgrade on lithium - T136312 [production]
14:28 <elukey> upgrade druid on druid100[4-6] to Druid 0.12.3 [production]
14:20 <banyek> running dns update (gerrit patch: 467711) [production]
13:48 <anomie@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Setting comment table migration stage to write-new/read-both on all wikis (T166733) (duration: 00m 55s) [production]
13:46 <godog> reformat ms-be2043 xfs filesystems - T199198 [production]
13:29 <XioNoX> test successful, rollback add term return-tcp permit on cr2-codfw [production]
13:28 <XioNoX> test add term return-tcp permit on cr2-codfw [production]
12:14 <volans> rebooting cumin1001 to pick new kernel and clear any potential weird state after OOMs [production]
12:01 <zeljkof> EU SWAT finished [production]
11:17 <zfilipin@deploy1001> Synchronized wmf-config/throttle.php: SWAT: [[gerrit:469261|New throttle rule for Johannesburg Event on 2018-10-27 (T207742)]] (duration: 00m 55s) [production]
11:08 <zfilipin@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:465418|Stop collecting data CitaitonUsage and CitationUsagePageLoad (T191086 T203253)]] (duration: 00m 57s) [production]
10:57 <volans> restart pdfrender on scb1003 [production]
10:11 <elukey> upgrade druid100[1-3] to druid 0.12.3 [production]
09:51 <gehel> resetting deployment directory on wdqs1003 [production]
09:15 <elukey@deploy1001> Finished deploy [analytics/turnilo/deploy@84bf1ad]: Upgrade to 1.8.1 (duration: 00m 10s) [production]
09:15 <elukey@deploy1001> Started deploy [analytics/turnilo/deploy@84bf1ad]: Upgrade to 1.8.1 [production]
09:10 <ema> resume cache hosts rolling reboots for kernel/microcode updates T203011 [production]
07:16 <vgutierrez> Uploaded certcentral 0.3 to apt.wikimedia.org (stretch) - T207737 T207478 [production]
07:11 <moritzm> installing requests security updates on trusty [production]
06:17 <SMalyshev> depooling wdqs1003 again, it's not catching up like the other hosts [production]
06:06 <elukey> upload druid 0.12.3-1 debs to stretch-wikimedia [production]
2018-10-24 §
23:24 <maxsem@deploy1001> Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/operations/mediawiki-config/+/469495/ (duration: 00m 54s) [production]
23:15 <maxsem@deploy1001> Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/operations/mediawiki-config/+/462040/ (duration: 00m 55s) [production]
23:08 <bawolff@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Deploy csp report-only to small.dblist wikis T207900 (duration: 00m 56s) [production]
22:38 <bawolff@deploy1001> Synchronized wmf-config/CommonSettings.php: Deploy csp report-only to outreachwiki T207900 (duration: 00m 54s) [production]
22:36 <bawolff@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Deploy csp report-only to outreachwiki T207900 (duration: 00m 54s) [production]
22:33 <bawolff@deploy1001> scap failed: average error rate on 8/11 canaries increased by 10x (rerun with --force to override this check, see https://logstash.wikimedia.org/goto/db09a36be5ed3e81155041f7d46ad040 for details) [production]
22:27 <eileen_> civicrm revision changed from 1c0a1b2406 to 97506677e8, config revision is c0a8be03a1 [production]