301-350 of 10000 results (27ms)
2014-10-21 §
08:50 <akosiaris> temporarily killed icinga-wm [production]
04:21 <LocalisationUpdate> ResourceLoader cache refresh completed at Tue Oct 21 04:21:19 UTC 2014 (duration 21m 18s) [production]
03:06 <LocalisationUpdate> completed (1.25wmf4) at 2014-10-21 03:06:47+00:00 [production]
02:33 <LocalisationUpdate> completed (1.25wmf3) at 2014-10-21 02:33:35+00:00 [production]
02:26 <mutante> rebooting iron for upgrades [production]
02:12 <mutante> restarting Apache on strontium [production]
01:58 <mutante> restarting apache on puppetmaster, temp. stopping icinga-wm [production]
01:31 <K4-713> adjusted fraud filters on payments [production]
00:32 <ori> Synchronized wmf-config: Id01fe7aac: Turn off spammy message cache log (duration: 00m 05s) [production]
2014-10-20 §
23:56 <awight> updated payments to 4cf8eb06a4746478c6424648c94688bf460cf63d [production]
23:31 <springle> upgrade db1004 trusty and reboot [production]
23:20 <mutante> installing package upgrades on iron [production]
23:04 <demon> Synchronized wmf-config/CommonSettings-labs.php: no-op, for completeness (duration: 00m 05s) [production]
22:38 <awight|purgreen> updating payments config with French Snowflake hack [production]
22:37 <Tim> doing load testing on mw1189 [production]
22:28 <Tim> enabled puppet on mw1189 [production]
21:08 <cscott> redis-cli srem "deploy:ocg/ocg:minions" tantalum.eqiad.wmnet [production]
21:08 <bblack> baham running gdnsd-2.1.0 test pkg [production]
21:08 <bd808> Deployed iegreview 203d509 (Disable strict variables for twig) [production]
21:01 <cscott> updated OCG to version ea10c93aca9bc1cae34f284fd74bb05d4b6a8cc6 [production]
20:33 <jgage> ulsfo repooled in dns [production]
20:17 <jgage> ulsfo fpc 1 mic 1 card swap complete [production]
20:12 <jgage> beginning router card swap in ulsfo [production]
20:11 <subbu> deployed Parsoid version d4567e9f [production]
20:01 <paravoid> cr1-ulsfo: reenable ospf/ospf3 (GTT is stable) [production]
19:13 <mutante> importing schema, data, users into mysql for iegreview [production]
17:42 <apergos> all trusty and lucid hosts now running salt 2014.1.11 (this includes labcontrol2001, salt master for future codfw labs) [production]
16:48 <reedy> Synchronized wmf-config/interwiki.cdb: Updating interwiki cache (duration: 00m 14s) [production]
16:47 <reedy> rebuilt wikiversions.cdb and synchronized wikiversions files: orwikisource [production]
16:44 <reedy> Synchronized wmf-config/: (no message) (duration: 00m 15s) [production]
16:44 <reedy> Synchronized database lists: orwikisource (duration: 00m 13s) [production]
16:43 <cmjohnson> reseating pem3 cr2-eqiad [production]
16:43 <apergos> all precise hosts salt updated to 2014.1.11, this includes tin (deployment) and virt1000 (salt master for labs). Not updated: virt1006 (inaccessible) [production]
15:58 <anomie> Synchronized wmf-config: SWAT: Enable wgSecurePollUseNamespace for testwiki [[gerrit:167592]] (duration: 00m 10s) [production]
15:57 <anomie> Finished scap: (no message) (duration: 18m 22s) [production]
15:39 <anomie> Started scap: (no message) [production]
15:39 <anomie> Synchronized php-1.25wmf4/extensions/SecurePoll/: SWAT: Update SecurePoll for testing on testwiki [[gerrit:167586]] (duration: 00m 10s) [production]
15:20 <_joe_> disabling puppet on mw1189 to do some hhvm testing [production]
15:20 <anomie> Synchronized php-1.25wmf3/resources/lib/oojs-ui/: SWAT: OOJS-UI bug fixes [[gerrit:167344]] (duration: 00m 12s) [production]
15:19 <anomie> Synchronized php-1.25wmf3/extensions/VisualEditor/: SWAT: VE bug fixes [[gerrit:167344]] (duration: 00m 10s) [production]
15:08 <anomie> Synchronized php-1.25wmf4/extensions/VisualEditor/: SWAT: VE bug fixes [[gerrit:167577]] (duration: 00m 10s) [production]
14:26 <paravoid> cr1-ulsfo: deactivating ospf/ospf3 on GTT ulsfo-eqiad link [production]
13:26 <paravoid> cr2-ulsfo: "request chassis mic {off,on}line fpc-slot 1 mic-slot 1" to reboot broken card [production]
12:28 <apergos> upgraded salt master (plus minion) on palladium to 2014.1.11, all neww precise installs will get this version now, other minion upgrades to follow shortly [production]
11:51 <godog> temporarily stopped ircecho/icinga-wm on neon, shower of alarms [production]
11:42 <godog> killed stray/old copy of diamond that was filling up conntrack on virt1000 [production]
09:53 <akosiaris> restarted ocg on ocg1001, ocg1002, ocg1003 [production]
07:07 <_joe_> rolling restart of ocg services [production]
04:29 <springle> removed old /var/log/ocg* on ocg1001 and ocg1003 and forced logrotate, / space critical [production]
03:42 <LocalisationUpdate> ResourceLoader cache refresh completed at Mon Oct 20 03:42:24 UTC 2014 (duration 42m 23s) [production]