9151-9200 of 10000 results (46ms)
2014-01-22 §
19:40 <matanya> git.wikimedia.org is down [production]
19:32 <reedy> synchronized wmf-config/ [production]
17:40 <RobH> virt0 puppet agent re-enabled and resumed normal service [production]
17:05 <RobH> issues with change, don't re-enable puppet agent on virt0 yet [production]
16:59 <RobH> rolling certificate changes to virt1000 and virt0. virt0 puppet agent disabled for moment [production]
16:33 <Nemo_bis> still no data in Global_JobQueue_length ganglia graph on hume (3 days now) [production]
14:34 <csteipp> synchronized php-1.23wmf10/includes/media 'bug60339' [production]
14:33 <csteipp> synchronized php-1.23wmf11/includes/media 'bug60339' [production]
11:18 <apergos> powercycled kaulen, it had died the swap death [production]
10:02 <apergos> restarted mw1206 a little while ago, it was inaccessible even via mgmt [production]
06:31 <reedy> synchronized wmf-config/InitialiseSettings.php [production]
05:37 <ori> restarting hafnium for kernel upgrade [production]
03:33 <LocalisationUpdate> ResourceLoader cache refresh completed at 2014-01-22 03:33:21+00:00 [production]
00:55 <marktraceur> deployed config for jsduck jobs for MultimediaViewer [production]
00:22 <demon> synchronized php-1.23wmf11/extensions/CirrusSearch 'Cirrus to master, with better job queues' [production]
00:21 <demon> synchronized php-1.23wmf10/extensions/CirrusSearch 'Cirrus to master, with better job queues' [production]
00:18 <demon> synchronized php-1.23wmf10/extensions/Elastica 'Elastica to master' [production]
00:15 <demon> synchronized php-1.23wmf10/extensions/CirrusSearch 'Rolling back Cirrus -- fatals' [production]
00:11 <demon> synchronized php-1.23wmf10/extensions/CirrusSearch 'Cirrus to master, with better job queues' [production]
2014-01-21 §
23:55 <marktraceur> deployed documentation jenkins jobs for MultimediaViewer [production]
23:52 <awight> crm updated from e82d2c5e64e67bee9bfa1f703fc94602addea2cb to f71526fa021fe3963a5d94151881f5a8a5c40dab [production]
20:24 <cmjohnson1> removing iptable rules for loudon and erzurumi [production]
20:20 <awight> updated listeners from dc21b3bf6560402238f9093f1dfbafe22be0a511 to 5e18193c5bb703e44c5cda22009ca97c295973e3 [production]
20:06 <awight> updated listeners from 888561ae0ee5a1d5f86da98ab3062f674726717f to dc21b3bf6560402238f9093f1dfbafe22be0a511 [production]
19:51 <awight> updated listeners from 7c5396c5954e28fe58033008a3b34ebf076a39a5 to 888561ae0ee5a1d5f86da98ab3062f674726717f [production]
19:42 <ori> synchronized wmf-config/CommonSettings.php 'Ie9ff56755: Add wmgUniversalLanguageSelectorDefault (2/2)' [production]
19:42 <ori> synchronized wmf-config/InitialiseSettings.php 'Ie9ff56755: Add wmgUniversalLanguageSelectorDefault' [production]
19:41 <ori> updated /a/common to {{Gerrit|Ie9ff56755}}: Add 'wmgUniversalLanguageSelectorDefault' [production]
19:17 <ori> synchronized php-1.23wmf10/extensions/UniversalLanguageSelector/Resources.php 'I05c76e478: Make ext.uls.mediawiki depend upon ext.uls.init' [production]
19:08 <ori> sync-file: mw1206: ssh: connect to host mw1206 port 22: No route to host [production]
19:07 <ori> synchronized php-1.23wmf11/extensions/UniversalLanguageSelector/Resources.php 'I05c76e478: Make ext.uls.mediawiki depend upon ext.uls.init' [production]
18:58 <awight> updated SmashPig to 43a27bd379632792bd7fc3ffc26c859d6dbbe0d6 [production]
17:06 <demon> finished scap: No code updates, just rebuilding all i18n (duration: 58m 29s) [production]
16:13 <demon> started scap: No code updates, just rebuilding all i18n [production]
11:59 <twkozlowski> bits application servers back to normal since around 11:46 UTC [production]
11:32 <ori> bits application server overload due to I71b70d8ee [production]
11:25 <ori> synchronized php-1.23wmf10/extensions/UniversalLanguageSelector 'I71b70d8ee: Add user preference to enable ULS' [production]
11:24 <ori> synchronized php-1.23wmf11/extensions/UniversalLanguageSelector 'I71b70d8ee: Add user preference to enable ULS' [production]
08:19 <andrewbogott> For wikitech labs interface: deployed new servicegroup schema and supporting code; ran maintenance/transitionServiceGroupSchema.php [production]
03:47 <ori> synchronized php-1.23wmf10/extensions/TimedMediaHandler/TimedMediaHandler.hooks.php 'Update TimedMediaHandler for I7a6da6c62' [production]
03:46 <ori> synchronized php-1.23wmf11/extensions/TimedMediaHandler/TimedMediaHandler.hooks.php 'Update TimedMediaHandler for I7a6da6c62' [production]
02:50 <LocalisationUpdate> ResourceLoader cache refresh completed at 2014-01-21 02:50:24+00:00 [production]
02:29 <ori> restarting gitblit using new upstart job def (I3f32dedf1) [production]
02:28 <LocalisationUpdate> completed (1.23wmf11) at 2014-01-21 02:28:40+00:00 [production]
02:15 <LocalisationUpdate> completed (1.23wmf10) at 2014-01-21 02:15:20+00:00 [production]
2014-01-20 §
13:51 <akosiaris> finished virt1000 reinitalization from virt0 [production]
13:51 <akosiaris> started virt1000 reinitialization from virt0 [production]
13:44 <akosiaris> stopped, backed up virt1000 and virt0 and started them up. Preparing for reinitializing virt1000 from virt0 [production]
11:13 <akosiaris> ms-be1003 kernel log show first lockups around Jan 19 06:30 UTC being XFS related. [production]
11:00 <ori> restarting Gitblit on antimony to test upstart script [production]