4251-4300 of 10000 results (54ms)
2018-10-25 ยง
23:39 <maxsem@deploy1001> Synchronized php-1.33.0-wmf.1/extensions/CentralNotice/: https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/CentralNotice/+/469794/ (duration: 00m 57s) [production]
23:26 <maxsem@deploy1001> Synchronized php-1.33.0-wmf.1/extensions/GlobalPreferences/: https://gerrit.wikimedia.org/r/c/469793/ (duration: 00m 58s) [production]
21:35 <twentyafterfour@deploy1001> rebuilt and synchronized wikiversions files: rolling back group1 refs T206655 T208000 [production]
21:29 <XioNoX> configure 208.80.153.185/29 on cr1/2-codfw - T207663 [production]
21:25 <twentyafterfour@deploy1001> rebuilt and synchronized wikiversions files: group2 wikis to 1.33.0-wmf.1 refs T206655 [production]
20:48 <twentyafterfour> staying at group1, error rate seems to have stabilized [production]
20:43 <ejegg> updated fundraising python tools from 5a2d39b41b to af5dbee8eb [production]
20:37 <ejegg> updated standalone SmashPig deployment from b638ca02bc to f65daa8550 [production]
20:32 <twentyafterfour> db error rate increased again. rolling back [production]
20:31 <twentyafterfour@deploy1001> Synchronized php: group1 wikis to 1.33.0-wmf.1 refs T206655 (duration: 00m 54s) [production]
20:30 <twentyafterfour@deploy1001> rebuilt and synchronized wikiversions files: group1 wikis to 1.33.0-wmf.1 refs T206655 [production]
20:07 <jgleeson_> Updated paymentswiki from 5a7a8e7e4f to f5999d963d [production]
20:04 <twentyafterfour> still haven't deployed wmf.1 yet error rate increased and icinga is alerting about mediawiki exceptions + wdqs1010 degraded [production]
20:02 <twentyafterfour@deploy1001> Finished scap: full sync to be sure that 1.33.0-wmf.1 is fully deployed (duration: 36m 57s) [production]
19:54 <mutante> mw1272 - repooled (T207983) [production]
19:51 <mutante> mw1272 - rebooting (a stop job is running for HHVM PH/Hack runtime) (T207983) [production]
19:47 <mutante> mw1272 - depooled, restarting hhvm (T207983) [production]
19:45 <mutante> mw1272 - depooled [production]
19:26 <twentyafterfour@deploy1001> Started scap: full sync to be sure that 1.33.0-wmf.1 is fully deployed [production]
19:23 <twentyafterfour> beginning mediawiki train. Will start with group1 and then monitor the situation for a few minutes. If everything looks good then we go to group2. [production]
19:20 <sbisson@deploy1001> Synchronized php-1.32.0-wmf.26/extensions/ContentTranslation/: SWAT: [[gerrit:469603|Add detailed logging for AbuseFilter]] (duration: 00m 56s) [production]
18:37 <sbisson@deploy1001> Synchronized php-1.33.0-wmf.1/extensions/ContentTranslation/: SWAT: [[gerrit:469605|Remove the session parameter from AbuseFilter logging]] (duration: 00m 56s) [production]
17:58 <aaron@deploy1001> Synchronized php-1.33.0-wmf.1/extensions/Translate/tag: c5fa239917a870240ec4dcd8a617f0f8033aa9bf (duration: 00m 55s) [production]
17:52 <aaron@deploy1001> Synchronized php-1.33.0-wmf.1/includes/page/WikiPage.php: f3b5a1df116f426c2809f2a266b9d761f15c349f (duration: 00m 54s) [production]
17:52 <smalyshev@deploy1001> Finished deploy [wdqs/wdqs@4967dba]: Test deploy new update & scripts (duration: 00m 28s) [production]
17:51 <smalyshev@deploy1001> Started deploy [wdqs/wdqs@4967dba]: Test deploy new update & scripts [production]
17:50 <aaron@deploy1001> Synchronized php-1.33.0-wmf.1/tests/phpunit/includes/page/WikiPageDbTestBase.php: f3b5a1df116f426c2809f2a266b9d761f15c349f (duration: 00m 55s) [production]
17:36 <aaron@deploy1001> Synchronized php-1.33.0-wmf.1/includes/changetags/ChangeTags.php: 08f8e6a9d7f1dcb281321c5e3a3471169e68348d (duration: 00m 55s) [production]
17:24 <bsitzmann@deploy1001> Finished deploy [mobileapps/deploy@95452cf]: Update mobileapps to 58cbdff (T206527) (duration: 03m 50s) [production]
17:20 <bsitzmann@deploy1001> Started deploy [mobileapps/deploy@95452cf]: Update mobileapps to 58cbdff (T206527) [production]
17:20 <mutante> planet - regenerating feeds for 'en' and 'de', others will follow by cron. switching to new theme. replaced bootstrap with bulma. removed jQuery. thanks to paladox [production]
16:34 <gehel@puppetmaster1001> conftool action : set/weight=20; selector: dc=eqiad,cluster=wdqs,name=wdqs1005.codfw.wmnet [production]
16:34 <gehel@puppetmaster1001> conftool action : set/weight=20; selector: dc=eqiad,cluster=wdqs,name=wdqs1004.codfw.wmnet [production]
16:34 <gehel> decreasing relative weight of wdqs1003 in LVS to ease the updater [production]
16:24 <shdubsh> installed patched nagios-nrpe-plugin and nagios-nrpe-server on icinga1001 - T207775 [production]
15:36 <elukey> shutdown aqs1006 to replace one broken disk - T206915 [production]
15:31 <SMalyshev> depooling wdqs1003 again, it's not catching up like the other hosts [production]
15:16 <addshore@deploy1001> Synchronized wmf-config/InitialiseSettings-labs.php: [Beta Cluster] Re-enable WBMI on Beta Commons (duration: 00m 54s) [production]
15:11 <addshore@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Revert "logging: Disable Wikibase.NewItemIdFormatter channel" (duration: 00m 55s) [production]
15:08 <addshore@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Explicitly set wgLexemeEnableRepo for wikidatas [[gerrit:469625]] (duration: 00m 55s) [production]
15:02 <godog> test rsyslog 8.38 upgrade on lithium - T136312 [production]
14:28 <elukey> upgrade druid on druid100[4-6] to Druid 0.12.3 [production]
14:20 <banyek> running dns update (gerrit patch: 467711) [production]
13:48 <anomie@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Setting comment table migration stage to write-new/read-both on all wikis (T166733) (duration: 00m 55s) [production]
13:46 <godog> reformat ms-be2043 xfs filesystems - T199198 [production]
13:29 <XioNoX> test successful, rollback add term return-tcp permit on cr2-codfw [production]
13:28 <XioNoX> test add term return-tcp permit on cr2-codfw [production]
12:14 <volans> rebooting cumin1001 to pick new kernel and clear any potential weird state after OOMs [production]
12:01 <zeljkof> EU SWAT finished [production]
11:17 <zfilipin@deploy1001> Synchronized wmf-config/throttle.php: SWAT: [[gerrit:469261|New throttle rule for Johannesburg Event on 2018-10-27 (T207742)]] (duration: 00m 55s) [production]