3301-3350 of 10000 results (16ms)
2015-08-03 §
09:31 <jynus> Synchronized wmf-config/db-eqiad.php: depool db1035 for maintenance (duration: 00m 12s) [production]
05:22 <@tin> ResourceLoader cache refresh completed at Mon Aug 3 05:22:15 UTC 2015 (duration 22m 14s) [production]
02:23 <@tin> LocalisationUpdate completed (1.26wmf16) at 2015-08-03 02:23:21+00:00 [production]
02:20 <l10nupdate> Synchronized php-1.26wmf16/cache/l10n: (no message) (duration: 06m 21s) [production]
01:47 <springle> starting OSC gerrit 228756 s5 wb_items_per_site.ips_site_page [production]
00:03 <krenair> Synchronized wmf-config/CommonSettings.php: https://gerrit.wikimedia.org/r/#/c/228198/ (duration: 00m 12s) [production]
2015-08-02 §
17:52 <ori> Synchronized wmf-config/InitialiseSettings.php: If7fcb6e6: Default wikipedias to enwiki.png (duration: 00m 12s) [production]
13:26 <jynus> powercycling analytics1044: same kernel fatal issues as 1043 [production]
13:10 <jynus> powercycling analytics1043: kernel issues [production]
12:05 <bblack> started pybal on lvs3001 [production]
04:56 <@tin> ResourceLoader cache refresh completed at Sun Aug 2 04:56:29 UTC 2015 (duration 56m 28s) [production]
02:23 <@tin> LocalisationUpdate completed (1.26wmf16) at 2015-08-02 02:23:09+00:00 [production]
02:20 <l10nupdate> Synchronized php-1.26wmf16/cache/l10n: (no message) (duration: 06m 11s) [production]
2015-08-01 §
06:04 <_joe_> removing some old apache access logs from mw1114 [production]
05:06 <@tin> ResourceLoader cache refresh completed at Sat Aug 1 05:06:46 UTC 2015 (duration 6m 45s) [production]
03:53 <andrewbogott> cleared out nova-conductor.log on labcontrol1001, restarted nova-conductor, graceful’d apache [production]
02:23 <@tin> LocalisationUpdate completed (1.26wmf16) at 2015-08-01 02:23:15+00:00 [production]
02:20 <l10nupdate> Synchronized php-1.26wmf16/cache/l10n: (no message) (duration: 06m 11s) [production]
00:12 <ori> Synchronized extract2.php: Ie919881a4: Add an API listing template to the allowed templates in extract2.php [production]
00:01 <ori> Synchronized php-1.26wmf16/includes: Revert I4afaecd8: "Avoiding writing sessions for no reason", and undo several uncommitted live-hacks for debugging T102199 (duration: 00m 16s) [production]
2015-07-31 §
20:14 <ori> Synchronized php-1.26wmf16/includes/objectcache/ObjectCacheSessionHandler.php: Uncommitted revert of I4afaecd to test impact on T102199 (duration: 00m 12s) [production]
20:11 <godog> revert to openjdk8 and restart cassandra on restbase1008 [production]
19:55 <ori> Synchronized php-1.26wmf16/includes/User.php: More debug logging for T102199 (duration: 00m 13s) [production]
19:54 <godog> revert to openjdk8 and restart cassandra on restbase1007 [production]
19:51 <ori> Synchronized php-1.26wmf16/includes/EditPage.php: More debug logging for T102199 (duration: 00m 12s) [production]
19:21 <godog> revert to openjdk8 and restart cassandra on restbase1006 [production]
19:02 <godog> revert to openjdk8 and restart cassandra on restbase1005 [production]
18:44 <twentyafterfour> oddly, the symptom was that there were logs about apc cache entries that had been on the GC queue for too long, I guess this is due to phd being stuck [production]
18:43 <twentyafterfour> restarted phd on iridium. I had to forcefully kill one stuck repository worker to get the daemons to restart properly. [production]
18:36 <godog> revert to openjdk8 and restart cassandra on restbase1004 [production]
18:15 <mutante> multatuli - installing package upgrades [production]
18:08 <legoktm> made User:Flow talk page manager a 'bot' on all wikis (except loginwiki) [production]
18:08 <godog> revert to openjdk8 and restart cassandra on restbase1003 [production]
17:53 <godog> revert to openjdk8 and restart cassandra on restbase1002 [production]
17:41 <godog> revert to openjdk8 and restart cassandra on restbase1001 T104887 [production]
17:11 <greg-g> follow on to previous to be explicit: it's not deployed, it is queued for Monday morning SWAT [production]
17:10 <aude> wmf/1.26wmf16 core submodule bump for Ic25edf7 (MultimediaViewer) is now on tin [production]
17:06 <aude> Synchronized php-1.26wmf16/extensions/Wikidata: Fix api xml format (duration: 00m 20s) [production]
15:52 <bd808> Rebuilt grafana-dashboards index to have 1 shard/2 replicas in logstash cluster [production]
15:46 <bd808> Rebuilt kibana-int index to have 1 shard/2 replicas in logstash cluster [production]
15:45 <andrewbogott> rebooting labvirt1005, again (3.16 this time) [production]
15:19 <jynus> Synchronized wmf-config/db-eqiad.php: reverting db1035 load to 10% (duration: 00m 14s) [production]
15:03 <urandom> bouncing restbase1005 (attempting to reproduce GC trends) [production]
14:54 <Coren> turned on alerting of backup status on labstore* with (by design) low limits. Expect alarms, and ignore. [production]
14:44 <kart_> Update cxserver to 9669e19 [production]
14:38 <andrewbogott> bumped the kernel version on labvirt1005, rebooting. [production]
14:09 <godog> restart cassandra on restbase1004 to apply java downgrade, missed from batch downgrade yesterday [production]
12:10 <godog> restbase1008 bootstrap finished successfully [production]
10:30 <jynus> Synchronized wmf-config/db-eqiad.php: returning db1035 to 100% load (duration: 00m 12s) [production]
08:19 <ori> Synchronized wmf-config/CommonSettings.php: I7be6dd2f5: Set $wgAjaxEditStash to false, on suspicion of being implicated in T102199 (duration: 00m 12s) [production]