501-550 of 10000 results (28ms)
2014-10-16 §
04:24 <LocalisationUpdate> ResourceLoader cache refresh completed at Thu Oct 16 04:24:21 UTC 2014 (duration 24m 20s) [production]
02:46 <LocalisationUpdate> completed (1.25wmf3) at 2014-10-16 02:46:46+00:00 [production]
02:25 <LocalisationUpdate> completed (1.25wmf2) at 2014-10-16 02:24:55+00:00 [production]
01:31 <springle> Synchronized wmf-config/db-eqiad.php: depool db1061 (duration: 00m 07s) [production]
00:49 <Krinkle> Zuul queue made unstuck by fixing the clogged build (see bug 72113) [production]
00:14 <Krinkle> Jenkins queue is stuck (99% free executors, but it's not running any of Zuul's pending jobs) [production]
2014-10-15 §
23:10 <maxsem> Synchronized php-1.25wmf2/extensions/MobileFrontend/: (no message) (duration: 00m 04s) [production]
23:04 <maxsem> Synchronized wmf-config: https://gerrit.wikimedia.org/r/#/c/166886/ (duration: 00m 05s) [production]
21:37 <ori> restarting hhvm on mw1114, this time with luasandbox [production]
21:20 <hoo> Gracefulled apache on mw1115 [production]
21:06 <ejegg> updated crm from 05e5388df34059c651223d53fb2986ac1c39a2d9 to d6a75b6df4482de61da372fa653902db7ca12766 [production]
21:02 <legoktm_> running mwscript extensions/CentralAuth/maintenance/migrateAccount.php on terbium for broken accounts (bug 61876) [production]
20:49 <mutante> deleting/shredding ishmael cert/keys from neon [production]
20:41 <hoo> Deleted 147 orphan wb_terms entries (bug 71914) [production]
20:40 <ori> restarting apache on mw1115 to test luasandbox [production]
20:25 <yurik> Synchronized wmf-config/mobile.php: Disable font for ZeroBanner (duration: 01m 05s) [production]
20:17 <hoo> Deleted ten orphan wb_entity_per_page rows on wikidata [production]
20:17 <yurik> Synchronized php-1.25wmf3/extensions/ZeroPortal: updatidng ZeroPortal to master (duration: 01m 11s) [production]
20:13 <arlolra> restarted ocg service [production]
20:02 <yurik> Synchronized php-1.25wmf2/extensions/ZeroPortal: updatidng ZeroPortal to master (duration: 01m 15s) [production]
19:47 <yurik> Synchronized php-1.25wmf3/extensions/ZeroBanner/: Latest ZeroBanner (duration: 01m 11s) [production]
19:37 <yurik> Synchronized php-1.25wmf2/extensions/ZeroBanner/: Latest ZeroBanner (duration: 01m 07s) [production]
19:05 <ottomata> deployed webstatscollector 0.4 on oxygen (filter) and gadolinium (collector) [production]
18:46 <ori> adjusting pybal weight for mw1114 back up to 20 to confirm that leak is in luasandbox [production]
17:57 <ori_> installed lua5.1 on mw1114 so i can switch scribunto to luastandalone and thus potentially isolate the leak to luasandbox [production]
15:38 <andrewbogott> running sync-common on virt1000 [production]
15:36 <marktraceur> Synchronized php-1.25wmf3/extensions/OpenStackManager/: [SWAT] [wmf3] Make list=novainstances available to anons (duration: 00m 06s) [production]
15:36 <marktraceur> Synchronized php-1.25wmf2/extensions/OpenStackManager/: [SWAT] [wmf2] Make list=novainstances available to anons (duration: 00m 05s) [production]
15:22 <paravoid> AMS-IX renumbering: remove old IP from interface, migration over; > 75% of total peers migrated, accounting for much more bandwidth/routes [production]
15:13 <marktraceur> Synchronized php-1.25wmf3/extensions/VisualEditor/modules/ve-mw/ui: [SWAT] [wmf3] Update OOjs UI to v0.1.0-pre (d74a46ca6a) and VisualEditor-MediaWiki to Ie06056b (duration: 00m 05s) [production]
15:12 <marktraceur> Synchronized php-1.25wmf3/resources/lib/oojs-ui: [SWAT] [wmf3] Update OOjs UI to v0.1.0-pre (d74a46ca6a) and VisualEditor-MediaWiki to Ie06056b (duration: 00m 06s) [production]
14:32 <paravoid> AMS-IX renumbering: move all remaining ASNs to the new space [production]
14:20 <Coren> Not reimaging mw1035 after all; hhvm is in our base, killing our ramz. [production]
14:16 <paravoid> AMS-IX renumbering: peering with (renumbered) top-10 ASNs + ASNs with large number of prefixes [production]
14:12 <Coren> reimaging mw1035 for great justice!!! (HHVM) [production]
14:11 <_joe_> powercycling mw1205, down since this morning, console blank [production]
13:29 <Jeff_Green> dist-upgrade and reboot indium [production]
12:40 <hashar> disabled/reenabled gearman plugin at https://integration.wikimedia.org/ci/manage [production]
12:34 <hashar> Zuul frozen \\O/ [production]
11:38 <_joe_> depooling mw1114 again [production]
10:59 <paravoid> AMS-IX renumbering: adding second IP, peering with RS1 [production]
10:47 <_joe_> repooled mw1114 with reduced load, using jemalloc with prof_leak enabled for sampling. will depool again soon [production]
10:27 <_joe_> depooling mw1114, stopping puppet for debugging purposes [production]
09:24 <godog> enable container sync for commons containers [production]
08:09 <hashar> restarting Jenkins [production]
08:06 <hashar> Jenkins: upgrading Gearman plugin to Patchset 9 of https://review.openstack.org/#/c/125755/ [production]
07:52 <springle> ongoing schema changes rev_content_(model|format) multiple shards, ok to kill osc_host.sh jobs on terbium in emergency [production]
06:58 <_joe_> restarting hhvm on mw1114 to avoid memory exhaustion [production]
05:54 <K4-713> updated payments to 944596744e0de23fee098 [production]
03:34 <LocalisationUpdate> ResourceLoader cache refresh completed at Wed Oct 15 03:34:50 UTC 2014 (duration 34m 49s) [production]