401-450 of 10000 results (33ms)
2016-03-17 §
12:05 <_joe_> switching cxserver to codfw [production]
11:44 <moritzm> rebooting acamar for kernel upgrade [production]
11:41 <gehel> restarting elasticsearch server elastic1009.eqiad.wmnet [production]
11:33 <_joe_> switching citoid to use codfw as well [production]
11:31 <moritzm> rebooting install2001 for kernel upgrade [production]
11:11 <oblivian@tin> Synchronized wmf-config/ProductionServices.php: switching mediawiki to use restbase in eqiad again (duration: 00m 32s) [production]
11:04 <mobrovac> restbase restarted cassandra-a on restbase1007 [production]
11:02 <gehel> restarting elasticsearch server elastic1008.eqiad.wmnet [production]
11:00 <mobrovac> restbase restarted cassandra on restbase2004 - OOM [production]
10:45 <moritzm> rebooting francium for kernel upgrade [production]
10:28 <oblivian@tin> Synchronized wmf-config/ProductionServices.php: switching mediawiki to use restbase in codfw (duration: 00m 32s) [production]
10:20 <moritzm> rebooting osmium for kernel upgrade [production]
10:14 <_joe_> external traffic is now flowing through restbase in codfw [production]
10:08 <_joe_> running puppet across eqiad varnishes to switch traffic of restbase eqiad => codfw [production]
09:34 <volans> Copying data from db2009 to db2008 T130098 [production]
09:02 <jynus> reimporting missing rows from production to labs (expect some lag during the day) [production]
08:59 <gehel> restarting elasticsearch server elastic1007.eqiad.wmnet [production]
08:44 <ori@tin> Synchronized php-1.27.0-wmf.16/includes/jobqueue/jobs/RefreshLinksJob.php: Revert: Job queue bankruptcy: force all refreshlinks jobs to be non-recursive (duration: 00m 29s) [production]
08:44 <ori@tin> Synchronized php-1.27.0-wmf.17/includes/jobqueue/jobs/RefreshLinksJob.php: Revert: Job queue bankruptcy: force all refreshlinks jobs to be non-recursive (duration: 00m 39s) [production]
08:26 <ori@tin> Synchronized php-1.27.0-wmf.17/includes/jobqueue/jobs/RefreshLinksJob.php: Job queue bankruptcy: force all refreshlinks jobs to be non-recursive (duration: 00m 25s) [production]
08:25 <ori@tin> Synchronized php-1.27.0-wmf.16/includes/jobqueue/jobs/RefreshLinksJob.php: Job queue bankruptcy: force all refreshlinks jobs to be non-recursive (duration: 00m 25s) [production]
07:48 <ori@tin> Synchronized php-1.27.0-wmf.16/includes/jobqueue/jobs/RefreshLinksJob.php: Job queue bankruptcy: force all refreshlinks jobs to be non-recursive (duration: 00m 24s) [production]
07:06 <ori> Short-circuiting RefreshLinksJob::run() to bail if the root job timestamp is older than March 1st [production]
06:46 <ori> Powercycled elastic1006; unresponsive. [production]
06:11 <ebernhardson> rebooting elastic1006.eqiad.wmnet for kernel update [production]
03:22 <l10nupdate@tin> ResourceLoader cache refresh completed at Thu Mar 17 03:22:55 UTC 2016 (duration 9m 33s) [production]
03:14 <krenair@tin> Synchronized wmf-config/InitialiseSettings-labs.php: https://gerrit.wikimedia.org/r/277452 (labs only change, just keeping file in sync) (duration: 00m 27s) [production]
03:13 <mwdeploy@tin> sync-l10n completed (1.27.0-wmf.17) (duration: 17m 44s) [production]
02:38 <mwdeploy@tin> sync-l10n completed (1.27.0-wmf.16) (duration: 17m 36s) [production]
00:57 <ebernhardson> rebooting elastic1005.eqiad.wmnet for kernel update [production]
00:05 <catrope@tin> Synchronized wmf-config/InitialiseSettings.php: Enable Flow by default in all talk namespaces on gomwiki (duration: 00m 28s) [production]
2016-03-16 §
23:56 <ebernhardson> rebooting elastic1004.eqiad.wmnet for kernel update [production]
23:47 <catrope@tin> Finished scap: bswiki namespace changes; MobileFrontend SWAT patches (duration: 05m 04s) [production]
23:42 <catrope@tin> Started scap: bswiki namespace changes; MobileFrontend SWAT patches [production]
23:15 <catrope@tin> Synchronized wmf-config/InitialiseSettings.php: Namespace config changes for bswiki (duration: 00m 44s) [production]
22:09 <ebernhardson> rebooting elastic1003.eqiad.wmnet for linux kernel upgrade [production]
21:47 <csteipp> deployed patch for T123071 [production]
21:18 <gehel> restarting elasticsearch server elastic1003.eqiad.wmnet [production]
20:06 <mutante> mira - git pull in mw-staging [production]
19:58 <twentyafterfour@tin> rebuilt wikiversions.php and synchronized wikiversions files: group1 wikis to 1.27.0-wmf.17 [production]
19:57 <gehel> regenerating puppet SSL certificates for elasticsearch eqiad cluster [production]
19:54 <mutante> rcs1001 - starting redis, disabling puppet (T130147) [production]
19:42 <mutante> rcs1002 - starting redis-server [production]
19:42 <mutante> rcs1001 - starting redis-server [production]
19:32 <twentyafterfour> Deploying 1.27.0-wmf.17 to group1 wikis [production]
19:31 <godog> restart pybal on lvs1005 T130143 [production]
19:18 <legoktm> created urlshortcodes table on wikishared db for UrlShortener [production]
19:16 <legoktm@tin> Synchronized wmf-config/CommonSettings.php: Configure UrlShortener extension in read-only mode (2/2) (duration: 00m 26s) [production]
19:15 <legoktm@tin> Synchronized wmf-config/InitialiseSettings.php: Configure UrlShortener extension in read-only mode (1/2) (duration: 00m 28s) [production]
19:14 <aaron@tin> Synchronized php-1.27.0-wmf.17/includes/jobqueue/Job.php: 3da38ce (duration: 00m 37s) [production]