401-450 of 10000 results (40ms)
2017-04-13 §
04:12 <mutante> ms-be2023 icinga alerts, no more swift processes. cant ssh to it. attempt to power cycle. mgmt console enourmous spam of "rejecting I/O to offline device" [production]
01:58 <bblack@neodymium> conftool action : set/pooled=yes; selector: name=achernar.wikimedia.org,dc=codfw,cluster=dns,service=pdns_recursor [production]
00:36 <catrope@tin> Finished scap: Split RCFilters GuidedTour messages for ORES vs non-ORES (T162693) (duration: 53m 47s) [production]
2017-04-12 §
23:42 <catrope@tin> Started scap: Split RCFilters GuidedTour messages for ORES vs non-ORES (T162693) [production]
23:37 <catrope@tin> Synchronized php-1.29.0-wmf.20/extensions/MobileFrontend/: Log only infoboxes which are not a direct children of lead section (T149884) (duration: 01m 05s) [production]
23:35 <catrope@tin> Synchronized php-1.29.0-wmf.20/resources/src/mediawiki.widgets: Fix setDisabled in mw.widgets.Complex* (T162667) (duration: 00m 42s) [production]
23:32 <catrope@tin> Synchronized php-1.29.0-wmf.19/resources/src/mediawiki.widgets: Fix setDisabled in mw.widgets.Complex* (T162667) (duration: 00m 44s) [production]
23:25 <awight> rebuilt and reenabled process-control jobs [production]
23:20 <catrope@tin> Synchronized wmf-config/InitialiseSettings.php: Disable cross-wiki uploads to Commons (T162374) (duration: 00m 43s) [production]
23:19 <cwd> removed p-c crontab to stop all jobs [production]
23:15 <bblack@neodymium> conftool action : set/pooled=no; selector: name=achernar.wikimedia.org,dc=codfw,cluster=dns,service=pdns_recursor [production]
23:13 <catrope@tin> Synchronized wmf-config/InitialiseSettings.php: Enable wgCiteResponsiveReferences on cawiki (T161307) and bgwiki (T162145) (duration: 00m 44s) [production]
23:02 <bblack@neodymium> conftool action : set/pooled=yes; selector: name=acamar.wikimedia.org,dc=codfw,cluster=dns,service=pdns_recursor [production]
22:50 <bblack> acamar fixed up BIOS: HT disabled and power mgmt was set to PPW (DAPC) instead of PPW (OS) [production]
22:45 <bblack> downtiming acamar again to fixup bios stuff (HT at least) [production]
21:31 <Dereckson> Create Education Program tables on it.wikiversity (T162692) [production]
20:44 <legoktm@tin> Synchronized wmf-config/InitialiseSettings.php: Deploy Linter to all wikis - T148609 (duration: 00m 44s) [production]
20:42 <bblack@neodymium> conftool action : set/pooled=no; selector: name=acamar.wikimedia.org,dc=codfw,cluster=dns,service=pdns_recursor [production]
20:25 <mutante> planet2001 - manually updating all feeds to make it active (or would have to wait for crons) [production]
20:12 <ssastry@tin> Finished deploy [parsoid/deploy@323cebb]: Updating Parsoid to 75debae3 (duration: 09m 16s) [production]
20:07 <mutante> planet2001 - activating all the crons, making planet active/active eqiad/codfw [production]
20:03 <ssastry@tin> Started deploy [parsoid/deploy@323cebb]: Updating Parsoid to 75debae3 [production]
19:42 <bd808@tin> Synchronized wmf-config/mc.php: Revert "wikitech: Enable binary memcached protocol" (duration: 00m 43s) [production]
19:36 <bd808> testing stashbot post-restart [production]
19:33 <demon@tin> Synchronized wmf-config/mc.php: Bryan made me do it (duration: 00m 43s) [production]
19:05 <demon@tin> rebuilt wikiversions.php and synchronized wikiversions files: group1 to wmf.20 [production]
19:05 <XenoRyet> reverted SmashPig from aede2772341b321aacbd970ec2b68655b704b147 to ab52dbe6628af2ebc717603259b584031807f6fe [production]
19:05 <demon@tin> Synchronized php: symlink bump (duration: 00m 42s) [production]
19:04 <ejegg> updated payments-wiki from 0b396a397272ed36325fcd944a3d0ba3625c212c to 36f38f6c96887f328596a1b01ad6a03ef75a1b29 [production]
18:52 <XenoRyet> updated SmashPig from ab52dbe6628af2ebc717603259b584031807f6fe to aede2772341b321aacbd970ec2b68655b704b147 [production]
18:45 <thcipriani@tin> Synchronized php-1.29.0-wmf.20/extensions/MobileFrontend: SWAT: [[gerrit:347802|formatter: Increase log level of infobox message]] T149884 (duration: 00m 46s) [production]
18:44 <ppchelko@tin> Finished deploy [changeprop/deploy@e403f56]: Config: Send ORES precache requests to both DCs. Attempt #2. T159615 (duration: 01m 15s) [production]
18:43 <ppchelko@tin> Started deploy [changeprop/deploy@e403f56]: Config: Send ORES precache requests to both DCs. Attempt #2. T159615 [production]
18:38 <thcipriani@tin> Synchronized php-1.29.0-wmf.20/extensions/MobileFrontend: SWAT: [[gerrit:347886|formatter: Change log channel of infobox message]] T149884 (duration: 00m 46s) [production]
18:37 <ppchelko@tin> Finished deploy [changeprop/deploy@0a9a008]: Config: Send ORES precache requests to both DCs. T159615 (duration: 06m 53s) [production]
18:30 <ppchelko@tin> Started deploy [changeprop/deploy@0a9a008]: Config: Send ORES precache requests to both DCs. T159615 [production]
18:26 <thcipriani@tin> Synchronized php-1.29.0-wmf.20/extensions/MobileFrontend: SWAT: [[gerrit:347885|setMobileOptions at time of skin creation]] T125588 (duration: 00m 46s) [production]
18:18 <thcipriani@tin> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:347888|Tweak Russian logo wordmark]] T162036 PART II (duration: 00m 43s) [production]
18:16 <thcipriani@tin> Synchronized static/images/mobile/copyright/wikipedia-wordmark-ru.svg: SWAT: [[gerrit:347888|Tweak Russian logo wordmark]] T162036 PART I (duration: 00m 43s) [production]
16:46 <awight@tin> rebuilt wikiversions.php and synchronized wikiversions files: (no justification provided) [production]
16:37 <awight@tin> Synchronized php-1.29.0-wmf.20/extensions/FundraiserLandingPage: Fix for donatewiki T162716 (duration: 00m 45s) [production]
16:35 <awight@tin> Synchronized php-1.29.0-wmf.19/extensions/FundraiserLandingPage: Fix for donatewiki T162716 (duration: 00m 48s) [production]
15:53 <chasemp> remove 2fa for Freddy2001 on wikitech per T162772 [production]
14:31 <andrewbogott> running maintain-meta_p on labsdb1001/1003/1009/1010/1011 [production]
14:23 <hashar> Restarting Jenkins for git/scm plugins updates [production]
14:06 <hashar> European SWAT complete [production]
13:51 <switchdc> (volans@neodymium) END TASK - switchdc.stages.t05_switch_traffic(codfw, eqiad) Successfully completed [production]
13:48 <switchdc> (volans@neodymium) START TASK - switchdc.stages.t05_switch_traffic(codfw, eqiad) Switch traffic flow to the appservers in the new datacenter [production]
13:42 <volans> testing t05_switch_traffic of the switchdc [production]
13:41 <elukey> apply SLOWLOG RESET and CONFIG SET slowlog-max-len 100000 (prev value 10000, 10ms) to rdb1005:6380 to track down slow reqs - T125735 [production]