9651-9700 of 10000 results (50ms)
2014-05-07 §
05:59 <springle> powercycled unresponsive neon, swapdeath + oom killer [production]
03:12 <LocalisationUpdate> ResourceLoader cache refresh completed at Wed May 7 03:11:36 UTC 2014 (duration 11m 35s) [production]
02:28 <LocalisationUpdate> completed (1.24wmf3) at 2014-05-07 02:27:23+00:00 [production]
02:15 <LocalisationUpdate> completed (1.24wmf2) at 2014-05-07 02:14:44+00:00 [production]
00:02 <mutante> upgrading libtiff on imagescalers [production]
2014-05-06 §
23:19 <maxsem> synchronized wmf-config/InitialiseSettings.php 'https://gerrit.wikimedia.org/r/131855' [production]
21:57 <mutante> gracefull'ing apaches [production]
20:43 <demon> synchronized php-1.24wmf3/extensions/CirrusSearch 'Rolling Cirrus back to known-good state' [production]
20:42 <demon> synchronized php-1.24wmf2/extensions/CirrusSearch 'Rolling Cirrus back to known-good state' [production]
20:27 <demon> synchronized php-1.24wmf3/extensions/CirrusSearch/includes/Hooks.php 'Fix typehinting' [production]
20:26 <demon> synchronized php-1.24wmf2/extensions/CirrusSearch/includes/Hooks.php 'Fix typehinting' [production]
20:20 <demon> synchronized php-1.24wmf3/extensions/CirrusSearch/CirrusSearch.php 'I2638b695: fix for page moves' [production]
20:19 <demon> synchronized php-1.24wmf2/extensions/CirrusSearch/CirrusSearch.php 'I2638b695: fix for page moves' [production]
20:18 <demon> synchronized php-1.24wmf3/extensions/CirrusSearch/includes/Hooks.php 'I2638b695: fix for page moves' [production]
20:17 <demon> synchronized php-1.24wmf2/extensions/CirrusSearch/includes/Hooks.php 'I2638b695: fix for page moves' [production]
19:00 <mutante> enabling puppet on netmon1001 [production]
17:39 <hoo> Changed email for global account "ElphiBot". [production]
16:49 <mattflaschen> synchronized php-1.24wmf3/extensions/GettingStarted/ 'GettingStarted token and logging deployment' [production]
16:45 <mattflaschen> synchronized php-1.24wmf2/extensions/GettingStarted/ 'GettingStarted token and logging deployment' [production]
16:19 <hoo> Changed email for global account "Elph". [production]
15:42 <akosiaris> killed zuul processes on gallium and restarted the service [production]
09:16 <LocalisationUpdate> ResourceLoader cache refresh completed at Tue May 6 09:15:20 UTC 2014 [production]
09:08 <LocalisationUpdate> completed (1.22wmf15) at Tue May 6 09:07:45 UTC 2014 [production]
09:01 <faidon> synchronized wmf-config/squid.php 'add Swift to squid.php' [production]
08:59 <faidon> updated /a/common to {{Gerrit|Ica9086dcd}}: Add Swift frontends to squid.php [production]
08:45 <ottomata> re-enabling puppet agent on analytics1022; kafka broker is caught up there and is fully in all ISRs [production]
08:36 <ottomata> temporarily disabling puppet on analytics1026 to troubleshoot a camus import problem [production]
06:55 <springle> hammering dbstore1001 with dumps in screen session. ignore replag [production]
06:31 <ori> ..on vanadium [production]
06:31 <ori> deleting rotated logs in /var/log/eventlogging/archive that are older than 90 days [production]
04:47 <springle> mydumper/myloader clone db1042 to db1049 [production]
04:07 <springle> synchronized wmf-config/db-eqiad.php 'depool db1049 for maintenance' [production]
03:48 <LocalisationUpdate> ResourceLoader cache refresh completed at Tue May 6 03:47:41 UTC 2014 (duration 47m 40s) [production]
03:03 <LocalisationUpdate> completed (1.24wmf3) at 2014-05-06 03:02:03+00:00 [production]
02:28 <LocalisationUpdate> completed (1.24wmf2) at 2014-05-06 02:27:21+00:00 [production]
00:02 <ori> synchronized wmf-config/CommonSettings.php 'Update wgFlowCacheVersion to 4.2' [production]
2014-05-05 §
23:57 <ori> updated /a/common to {{Gerrit|Id1f2e0acf}}: Drop wgFlowCacheKey from CommonSettings.php [production]
23:54 <ori> Finished scap: SWAT deploy for VisualEditor and Flow cherry-picks (duration: 09m 55s) [production]
23:44 <ori> Started scap: SWAT deploy for VisualEditor and Flow cherry-picks [production]
23:26 <ori> synchronized php-1.24wmf3/extensions/EventLogging 'Update EventLogging for Id23b37fbe for SWAT.' [production]
23:23 <ori> synchronized php-1.24wmf2/extensions/EventLogging 'Update EventLogging for Id23b37fbe for SWAT.' [production]
21:07 <jgage> trying on analytics1022: https://wikitech.wikimedia.org/wiki/Analytics/Kraken/Kafka/Administration#Recovering_a_laggy_broker_replica [production]
20:58 <RobH> ssl1001-1003 now have updated unified cert in service [production]
20:58 <jgage> both kafka brokers back in service [production]
20:54 <RobH> cp4001-4020 unified cert and nginx service reloaded, back in service [production]
20:50 <RobH> ssl1006 and ssl1009 are responsive to nginx and back in service [production]
20:43 <RobH> pybal [production]
20:43 <RobH> ssl1009 was refusing connections both before and after my ssl cert update. ssl1006 is presently refusing connections post update. they are set to disabled in pubal [production]
20:40 <RobH> ssl1008 back into service, ssl1009 already depooled [production]
20:38 <jgage> forced kafka broker reelection [production]