2101-2150 of 10000 results (41ms)
2016-01-07 §
19:11 <jynus> setting up watchdog process killing long running queries on db1051 [production]
18:54 <_joe_> also resetting the drac [production]
18:53 <_joe_> powercycling ms-be1013 [production]
02:32 <l10nupdate@tin> l10nupdate@tin ResourceLoader cache refresh completed at Thu Jan 7 02:32:04 UTC 2016 (duration 6m 54s) [production]
02:25 <mwdeploy@tin> mwdeploy@tin sync-l10n completed (1.27.0-wmf.9) (duration: 10m 33s) [production]
2016-01-06 §
23:03 <gwicke> switched restbase1009 to node 4.2 for testing, and restarted restbase; see https://phabricator.wikimedia.org/T107762 [production]
02:34 <l10nupdate@tin> l10nupdate@tin ResourceLoader cache refresh completed at Wed Jan 6 02:34:38 UTC 2016 (duration 6m 53s) [production]
02:27 <mwdeploy@tin> mwdeploy@tin sync-l10n completed (1.27.0-wmf.9) (duration: 10m 30s) [production]
2016-01-05 §
22:38 <aaron@tin> aaron@tin Synchronized rpc: 830e1ed8d80295710dc02f18102b4fadae7fca86 (duration: 00m 55s) [production]
18:34 <jzerebecki@tin> jzerebecki@tin scap aborted: deploy-log (duration: 00m 04s) [production]
18:34 <jzerebecki@tin> jzerebecki@tin Started scap: deploy-log [production]
15:47 <ottomata> transitioned analytics1001 to active namenode [production]
03:51 <krinkle@tin> krinkle@tin Synchronized php-1.27.0-wmf.9/includes/specials/SpecialJavaScriptTest.php: Idaacf71870 (duration: 00m 30s) [production]
03:50 <krinkle@tin> krinkle@tin Synchronized php-1.27.0-wmf.9/resources/src/mediawiki.special/: Idaacf71870 (duration: 00m 30s) [production]
03:49 <krinkle@tin> krinkle@tin Synchronized php-1.27.0-wmf.9/resources/Resources.php: Idaacf71870 (duration: 00m 36s) [production]
02:31 <l10nupdate@tin> l10nupdate@tin ResourceLoader cache refresh completed at Tue Jan 5 02:31:46 UTC 2016 (duration 6m 54s) [production]
02:24 <mwdeploy@tin> mwdeploy@tin sync-l10n completed (1.27.0-wmf.9) (duration: 10m 13s) [production]
2016-01-04 §
20:50 <mutante> ms-be1011 - powercycled, was frozen [production]
20:43 <mutante> ms-be2007 - System halted!Error: Integrated RAID [production]
20:42 <mutante> ms-be2007 - powercycle (was status: on but all frozen) (i assume xfs like be2006 appears in SAL recently) [production]
20:36 <mutante> mw2019 - puppet run (icinga claimed it failed but just here) [production]
20:19 <mutante> rutherfordium - attempt to restart with gnt-instance [production]
20:12 <mutante> rutherfordium (people.wm) was down for days per icinga - then magically fixes itself when i connect to console but before even loggin in (ganeti VM) [production]
20:00 <mutante> mw1123 - start HHVM (was 503 and service stopped) [production]
19:28 <mutante> elastic1006 - out of disk - gzip eqiad_index_search_slowlog.log files [production]
17:37 <yurik@tin> yurik@tin Synchronized php-1.27.0-wmf.9/extensions/Graph/: Deployed Graph ext - gerrit 262357 (duration: 00m 33s) [production]
02:32 <l10nupdate@tin> l10nupdate@tin ResourceLoader cache refresh completed at Mon Jan 4 02:32:10 UTC 2016 (duration 6m 53s) [production]
02:25 <mwdeploy@tin> mwdeploy@tin sync-l10n completed (1.27.0-wmf.9) (duration: 10m 05s) [production]
2016-01-03 §
02:31 <l10nupdate@tin> l10nupdate@tin ResourceLoader cache refresh completed at Sun Jan 3 02:31:58 UTC 2016 (duration 6m 52s) [production]
02:25 <mwdeploy@tin> mwdeploy@tin sync-l10n completed (1.27.0-wmf.9) (duration: 10m 22s) [production]
2016-01-02 §
03:34 <twentyafterfour> deploying https://gerrit.wikimedia.org/r/261725, restarted apache2 on iridium [production]
02:31 <l10nupdate@tin> l10nupdate@tin ResourceLoader cache refresh completed at Sat Jan 2 02:31:28 UTC 2016 (duration 6m 58s) [production]
02:24 <mwdeploy@tin> mwdeploy@tin sync-l10n completed (1.27.0-wmf.9) (duration: 10m 09s) [production]
01:04 <YuviPanda> imported vagrant 1.8.1 for jessie per bd808 [production]
00:04 <ori> (at 23:46 UTC) restarted nova-compute on labvirt1002 [production]
2016-01-01 §
23:50 <legoktm> restarted nodepool on labnodepool1001 [production]
23:37 <ori> restarting nodepool on labnodepool1001.eqiad.wmnet (T122731) [production]
19:41 <bd808> Updated scholarships.wikimedia.org with latest translation data from translatewiki [production]
02:30 <l10nupdate@tin> l10nupdate@tin ResourceLoader cache refresh completed at Fri Jan 1 02:30:27 UTC 2016 (duration 6m 47s) [production]
02:23 <mwdeploy@tin> mwdeploy@tin sync-l10n completed (1.27.0-wmf.9) (duration: 09m 58s) [production]
2015-12-31 §
02:31 <l10nupdate@tin> ResourceLoader cache refresh completed at Thu Dec 31 02:31:43 UTC 2015 (duration 6m 52s) [production]
02:24 <mwdeploy@tin> sync-l10n completed (1.27.0-wmf.9) (duration: 10m 25s) [production]
2015-12-30 §
15:05 <andrewbogott> restarting puppet on seaborgium; openldap will restart but config should not change [production]
14:57 <andrewbogott> restarting puppet on serpens; openldap will restart but config should not change [production]
13:14 <mark> labstore1001: apt-get install irqbalance [production]
08:01 <jynus> setting dbstore1001 to read_only, converting ruwiki.recentchanges back to InnoDB [production]
02:32 <l10nupdate@tin> ResourceLoader cache refresh completed at Wed Dec 30 02:32:15 UTC 2015 (duration 6m 55s) [production]
02:26 <gwicke> restbase: finished full deploy of 7db8e216 (small bug fix & a security fix) to production cluster [production]
02:25 <mwdeploy@tin> sync-l10n completed (1.27.0-wmf.9) (duration: 09m 50s) [production]
02:17 <gwicke> restbase: starting full deploy of 7db8e216 (small bug fix & a security fix) to production cluster [production]