5901-5950 of 10000 results (37ms)
2015-09-28 §
20:51 <bearND> MobileApps deployed sha1 9df72ec [production]
20:42 <subbu> deployed parsoid version b9e5244e + hotfix on tin to turn off batching api use since canary restart of wtp1002 showed some batching api errors [production]
20:26 <yuvipanda> depooled cp2017 from pybal config too [production]
20:22 <yuvipanda> depooled cp2017 since it's down [production]
19:02 <chasemp> powercycle iridium via console as it's unresponsive [production]
15:50 <krenair@tin> Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/241579/ (duration: 00m 17s) [production]
15:28 <krenair@tin> Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/241598/ (duration: 00m 17s) [production]
15:11 <krenair@tin> Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/241649/ (duration: 00m 17s) [production]
14:00 <paravoid> running failover eqiad->codfw test for all frontend traffic [production]
06:11 <ori@tin> Synchronized wmf-config/CommonSettings.php: I179be4bd3: Rely on timeouts specified in php.ini rather than calling ini_set() (duration: 00m 17s) [production]
05:52 <l10nupdate@tin> ResourceLoader cache refresh completed at Mon Sep 28 05:52:34 UTC 2015 (duration 52m 33s) [production]
02:27 <l10nupdate@tin> LocalisationUpdate completed (1.26wmf24) at 2015-09-28 02:27:16+00:00 [production]
02:20 <l10nupdate@tin> Synchronized php-1.26wmf24/cache/l10n: l10nupdate for 1.26wmf24 (duration: 06m 54s) [production]
2015-09-27 §
23:17 <ori@tin> Synchronized php-1.26wmf24/./resources/src/mediawiki.toolbar/toolbar.less: I94ced06178: mediawiki.toolbar: temporary workaround for T113868 (duration: 00m 17s) [production]
16:51 <Krenair> ran sync-common on snapshot1001 to bring it up to date [production]
16:43 <krenair@tin> Purged l10n cache for 1.26wmf21 [production]
16:42 <krenair@tin> Purged l10n cache for 1.26wmf20 [production]
16:42 <krenair@tin> Purged l10n cache for 1.26wmf19 [production]
15:50 <krenair@tin> Synchronized wmf-config/abusefilter.php: https://gerrit.wikimedia.org/r/#/c/241354/ - fix AbuseFilter block durations (duration: 00m 18s) [production]
10:05 <jynus> leaving innodb compression tests on es2005 running (could affect lag on that host) [production]
02:23 <l10nupdate@tin> Synchronized php-1.26wmf24/cache/l10n: l10nupdate for 1.26wmf24 (duration: 07m 05s) [production]
2015-09-26 §
19:04 <hashar> restarting Jenkins. Just in case :-D [production]
02:23 <l10nupdate@tin> Synchronized php-1.26wmf24/cache/l10n: l10nupdate for 1.26wmf24 (duration: 06m 48s) [production]
2015-09-25 §
23:54 <ori@tin> Synchronized php-1.26wmf24/includes/GlobalFunctions.php: 37c6972f94: Made wfIsBadImage() use APC (duration: 00m 17s) [production]
22:16 <mutante> sodium - shutting down [production]
21:53 <ori> Armed new servicedeploy_rsa on mira [production]
21:51 <ori> Armed new servicedeploy_rsa on tin [production]
20:45 <urandom> starting a Cassandra repair on xeon (nodetool repair -pr) [production]
19:30 <mutante> restarted apache on gallium [production]
19:07 <bblack> configuring and enabling lvs-cross-row ports on asw-b-eqiad for lvs1007,8,10,11 [production]
18:04 <jynus> performing schema change on m5-master "nova" [production]
17:26 <jynus> restarting and upgrading db1051 mysql (depooled) [production]
17:20 <godog> reboot ms-be1005, xfs [production]
16:17 <akosiaris> duplicating database otrs to otrsupgradetest for testing the upgrade procedure [production]
16:16 <cmjohnson1> cp1046 stopped icinga checks for hardware troubleshooting [production]
14:43 <moritzm> installed rpcbind and apport security updates on various servers [production]
14:36 <krenair@tin> Synchronized php-1.26wmf24/includes/specials/SpecialMovepage.php: https://gerrit.wikimedia.org/r/#/c/241045/ (duration: 00m 17s) [production]
14:22 <chasemp> 'sudo -u hdfs hdfs haadmin -transitionToActive analytics1001-eqiad-wmnet' per otto on analytics1001 [production]
14:03 <urandom> starting a Cassandra repair on restbase1006 (nodetool repair -pr -dc eqiad) [production]
13:58 <hashar> nodepool back in operations [production]
13:44 <moritzm> restarted saltmaster on palladium [production]
13:12 <mobrovac> restbase deploying e42bf0fc [production]
13:12 <moritzm> added debdeploy 0.0.7 to carbon [production]
13:10 <godog> enable puppet on restbase in production [production]
12:54 <andrewbogott> restarted rabbitmq-server on labcontrol1001 [production]
12:52 <godog> stop puppet on restbase in production -- config deployment [production]
12:46 <hashar> stopping nodepool to clear out left over mysql connections [production]
12:21 <hashar> Nodepool is dead, or at least not adding new slaves anymore [production]
12:16 <hashar> Seems Zuul/Jenkins is in trouble somehow :-/ [production]
11:42 <andrewbogott> upgrading linux-image-generic on labnet1002 to get us away from the recently-crashed 3.13.0-59 [production]