3401-3450 of 10000 results (72ms)
2018-01-02 §
08:29 <_joe_> restarting hhvm on mw1280,1282 for the same reasons [production]
08:26 <_joe_> restarting hhvm on mw1317, multiple threads stuck in HPHP::jit::enterTCImpl [production]
08:23 <elukey> restart druid coordinators on druid* to pick up new jvm settings [production]
08:22 <elukey> restart druid coordinators to pick up new jvm settings (freeing up 6GB of used memory) [analytics]
08:19 <_joe_> restarting hhvm on mw1313, concurrency HPHP::VariableUnserializer::unserializeVariant [production]
08:06 <marostegui> Deploy alter table on db1039 (already depooled) - T174569 [production]
07:56 <marostegui> Deploy schema change on dbstore1001.s7 - T174569 [production]
06:51 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Repool db1110 (duration: 00m 52s) [production]
06:42 <marostegui> Stop db1110 and dbstore1002.s5 replication in sync [production]
06:28 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1110 to reimport dewiki.langlinks on dbstore1002 (duration: 00m 50s) [production]
2018-01-01 §
23:46 <Krenair> ran `mwscript extensions/ORES/maintenance/CheckModelVersions.php --wiki=sqwiki` on deployment-tin for T183862 [deployment-prep]
23:46 <Krenair> ran `mwscript extensions/ORES/maintenance/CheckModelVersions.php --wiki=sqwiki` on deployment-tin for T183862 [releng]
22:45 <eileen1> re-enable silverpop jobs process-control config revision is 4a4c61ae1b [production]
22:36 <eileen1> update process-control to process-control config revision is 65b65972a9 (silverpop password change) [production]
21:01 <addshore@tin> Synchronized php-1.31.0-wmf.12/extensions/WikimediaEvents: [[gerrit:399627|Add onBeforeInitializeWMDECampaign]] T182797 T182794 PT 2/2 (duration: 00m 53s) [production]
20:59 <addshore@tin> Synchronized php-1.31.0-wmf.12/extensions/WikimediaEvents/WikimediaEventsHooks.php: [[gerrit:399627|Add onBeforeInitializeWMDECampaign]] T182797 T182794 PT 1/2 (duration: 00m 55s) [production]
19:56 <Amir1> ladsgroup@deployment-tin:~$ mwscript extensions/ORES/maintenance/CheckModelVersions.php --wiki=nlwiki (T183862) [releng]
19:56 <Amir1> restarting ores services in deployment-sca03 (T183862) [releng]
06:35 <bd808> Does it still work? [tools.stashbot]
2017-12-31 §
02:00 <bd808> Killed some pwb.py and qacct processes running on tools-bastion-03 [tools]
00:42 <Sagan> Neon, Oxygen, Tin: Confirming current config is working as expected [rcm]
00:38 <Sagan> Neon: Testing current config by producing load on runners [rcm]
00:37 <Sagan> Neon: Setting new weight on runners. Tin = 4; Oxygen = 2 [rcm]
00:29 <Sagan> Tin and Oxygen: Recreating runners, four per server [rcm]
00:19 <Sagan> Tin: Rebooting [rcm]
2017-12-30 §
18:22 <ejegg> disabled failing silverpop data fetch jobs [production]
04:54 <Reedy> kill another long running high cpu usage process on tools-bastion-03 (of the same sga2.tcl file) [tools.giftbot]
04:45 <Reedy> kill long running high cpu usage process on tools-bastion-03 [tools.giftbot]
2017-12-29 §
18:31 <mutante> gerrit: restart service to apply change from Dec 20th, avoid logspam due to unapplied config change [production]
18:02 <Sagan> Neon: Package update [rcm]
15:38 <ariel@tin> Finished deploy [dumps/dumps@51d4fd6]: enable pageslogging dump in parallel jobs (duration: 00m 02s) [production]
15:38 <ariel@tin> Started deploy [dumps/dumps@51d4fd6]: enable pageslogging dump in parallel jobs [production]
14:46 <apergos> restarted puppetdb on nitrogen, puppetdb refusing to hand out facts again [production]
08:34 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Repool db1110 (duration: 00m 52s) [production]
08:19 <marostegui> Stop replication in sync on dbstore1002.s5 and db1110 [production]
08:03 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1110 to reimport dewiki.imagelinks on dbstore1002 (duration: 00m 52s) [production]
07:43 <marostegui> Fixing dbstore1002 dewiki.pagelinks [production]
00:08 <Sagan> Neon: Update Packages [rcm]
2017-12-28 §
14:18 <marostegui> Power cycle pc2005 as it is down [production]
2017-12-27 §
19:54 <godog> power reset ms-be1033 [production]
19:47 <godog> reboot ms-be1033 - T183724 [production]
19:42 <legoktm> legoktm@integration-slave-jessie-1003:/srv/jenkins-workspace/workspace$ sudo rm -rf * [releng]
17:00 <twentyafterfour> restarting apache on phab1001 [production]
15:54 <mobrovac@tin> Started restart [electron-render/deploy@94d27d7]: Bounce Electron, stuck - T174916 [production]
2017-12-26 §
16:33 <mobrovac@tin> Finished deploy [mathoid/deploy@63b2ddc]: Bring back Mathoid in codfw to v0.6.5 in sync with eqiad - T179419 T172767 (duration: 01m 44s) [production]
16:31 <mobrovac@tin> Started deploy [mathoid/deploy@63b2ddc]: Bring back Mathoid in codfw to v0.6.5 in sync with eqiad - T179419 T172767 [production]
16:27 <mutante> re-generated DNS zones to add new language lfn (Lingua Franca Nova) (T183561) [production]
12:00 <Amir1> ladsgroup@terbium:~$ mwscript extensions/Cognate/maintenance/populateCognatePages.php --wiki hifwiktionary (T180785) [production]
11:59 <Amir1> ladsgroup@terbium:~$ mwscript extensions/Cognate/maintenance/populateCognateSites.php --wiki aawiktionary --site-group wiktionary (T180785) [production]
11:05 <ariel@tin> Finished deploy [dumps/dumps@8c44d0a]: re-enable file size reporting for dump files still being written (duration: 00m 02s) [production]