7251-7300 of 10000 results (33ms)
2015-01-12 §
17:33 <hoo> mw1010: rsync: failed to set times on "/srv/mediawiki/.": Read-only file system (30) [production]
17:31 <hoo> Synchronized php-1.25wmf13/extensions/CentralAuth/: Only test passwords once in CentralAuthUser::prepareMigration - 2nd try (duration: 00m 07s) [production]
17:31 <hoo> Synchronized php-1.25wmf14/extensions/CentralAuth/: Only test passwords once in CentralAuthUser::prepareMigration (duration: 00m 06s) [production]
17:31 <hoo> Synchronized php-1.25wmf13/extensions/CentralAuth/: Only test passwords once in CentralAuthUser::prepareMigration (duration: 00m 06s) [production]
17:22 <mutante> restarted icinga-wm to join -releng [production]
17:02 <mutante> labmon1001 - purging mlocate package that was status 'rc' [production]
16:30 <godog> stop/start graphite-web on tungsten to clear logs [production]
16:28 <bd808> deleted 2014-01-* and 2015-12-* indices from logstash elasticsearch cluster [production]
16:13 <bd808> logs on logstash1001 reporting elasticserch connection errors; restarted logstash service [production]
16:10 <anomie> Synchronized wmf-config/InitialiseSettings.php: SWAT: Enable "Other projects sidebar" by default on frwiki [[gerrit:183288]] (duration: 00m 05s) [production]
16:09 <bd808> logstash elasticsearch cluster has strange indices dated 2014-01-* and 2015-12-* again [production]
16:05 <bd808> restarted elasticsearch on logstash1001 [production]
16:05 <anomie> Synchronized wmf-config/InitialiseSettings.php: SWAT: Disable thumbnail prerendering in production [[gerrit:183885]] (duration: 00m 06s) [production]
16:03 <_joe_> depooling mw1062, disk errors [production]
16:03 <bd808> elasticsearch on logstash1001 not responding to http requests [production]
16:01 <aude> Synchronized wmf-config/Wikibase.php: Enable usage tracking on test.wikidata and testwiki (duration: 00m 05s) [production]
16:00 <aude> Synchronized wmf-config/InitialiseSettings.php: (no message) (duration: 00m 05s) [production]
15:59 <bd808> logstash not showing any events at all since 2015-01-12T13:58:59.728Z [production]
15:58 <aude> Finished scap: Update Wikidata and WikimediaMessages (duration: 32m 06s) [production]
15:35 <hashar> Enabling test/gate of several extensions together. {{gerrit|180494}} , RFC extensions continuous integration {{bug|T1350}} [production]
15:27 <aude> Added and populated wbc_entity_usage table on testwiki and testwikidatawiki [production]
15:25 <aude> Started scap: Update Wikidata and WikimediaMessages [production]
15:10 <hashar> Synchronized php-1.25wmf13/extensions/Echo/tests/phpunit/includes/cache/TitleLocalCacheTest.php: php-1.25wmf14/extensions/Echo/tests/phpunit/includes/cache/TitleLocalCacheTest.php (duration: 00m 05s) [production]
15:02 <hashar> restarting Zuul. Deadlocked due to Gerrit database [production]
12:20 <_joe_> upgrading HHVM on the API cluster [production]
10:45 <_joe_> restarting hhvm on mw1126, stuck in HPHP::StatCache::refresh [production]
10:22 <_joe_> upgrading HHVM on all appservers [production]
10:11 <hashar> Synchronized wmf-config/throttle.php: Tel-Hai Academic College event - Bug: T85773 (duration: 00m 07s) [production]
09:23 <hashar> Restarting Zuul [production]
09:06 <hashar> Tweak Zuul configuration to pin python-daemon <= 2.0 and deploying tag wmf-deploy-20150112-1. {{bug|T86513}} [production]
07:24 <andrewbogott> on virt1005 and virt1006, ran 'ln -s /usr/bin/qemu-system-x86_64 /usr/bin/kvm' that allows nova to migrate instances between hosts. [production]
03:54 <LocalisationUpdate> ResourceLoader cache refresh completed at Mon Jan 12 03:54:34 UTC 2015 (duration 54m 33s) [production]
02:17 <LocalisationUpdate> completed (1.25wmf14) at 2015-01-12 02:17:48+00:00 [production]
02:17 <l10nupdate> Synchronized php-1.25wmf14/cache/l10n: (no message) (duration: 00m 01s) [production]
02:11 <LocalisationUpdate> completed (1.25wmf13) at 2015-01-12 02:10:54+00:00 [production]
02:10 <l10nupdate> Synchronized php-1.25wmf13/cache/l10n: (no message) (duration: 00m 01s) [production]
2015-01-11 §
16:44 <springle> db1050 dberror log noise was https://phabricator.wikimedia.org/T86482 [production]
16:29 <springle> Synchronized wmf-config/db-eqiad.php: repool db1050, warm up (duration: 00m 05s) [production]
16:08 <springle> Synchronized wmf-config/db-eqiad.php: depool db1050, mysqld got TERM somehow (duration: 00m 05s) [production]
03:59 <LocalisationUpdate> ResourceLoader cache refresh completed at Sun Jan 11 03:59:26 UTC 2015 (duration 59m 25s) [production]
02:18 <LocalisationUpdate> completed (1.25wmf14) at 2015-01-11 02:18:10+00:00 [production]
02:18 <l10nupdate> Synchronized php-1.25wmf14/cache/l10n: (no message) (duration: 00m 04s) [production]
02:11 <LocalisationUpdate> completed (1.25wmf13) at 2015-01-11 02:11:13+00:00 [production]
02:11 <l10nupdate> Synchronized php-1.25wmf13/cache/l10n: (no message) (duration: 00m 02s) [production]
2015-01-10 §
23:20 <_joe_> restarted the puppetmaster on palladium [production]
19:44 <hoo> Synchronized wmf-config/Bug54847.php: Don't use protected CentralAuthUser::getPasswordObject (duration: 00m 06s) [production]
18:18 <hoo> Synchronized wmf-config/Bug54847.php: Bug54847.php: Replace removed CentralAuthUser::getPasswordHash (duration: 00m 06s) [production]
18:10 <hoo> Ran mysql:wikiadmin@db1033 [centralauth]> DELETE FROM bug_54847_password_resets WHERE r_username = 'Tk'; [production]
04:03 <LocalisationUpdate> ResourceLoader cache refresh completed at Sat Jan 10 04:03:52 UTC 2015 (duration 3m 50s) [production]
02:24 <LocalisationUpdate> completed (1.25wmf14) at 2015-01-10 02:24:48+00:00 [production]