2016-03-17
§
|
11:31 |
<moritzm> |
rebooting install2001 for kernel upgrade |
[production] |
11:11 |
<oblivian@tin> |
Synchronized wmf-config/ProductionServices.php: switching mediawiki to use restbase in eqiad again (duration: 00m 32s) |
[production] |
11:04 |
<mobrovac> |
restbase restarted cassandra-a on restbase1007 |
[production] |
11:02 |
<gehel> |
restarting elasticsearch server elastic1008.eqiad.wmnet |
[production] |
11:00 |
<mobrovac> |
restbase restarted cassandra on restbase2004 - OOM |
[production] |
10:45 |
<moritzm> |
rebooting francium for kernel upgrade |
[production] |
10:28 |
<oblivian@tin> |
Synchronized wmf-config/ProductionServices.php: switching mediawiki to use restbase in codfw (duration: 00m 32s) |
[production] |
10:20 |
<moritzm> |
rebooting osmium for kernel upgrade |
[production] |
10:14 |
<_joe_> |
external traffic is now flowing through restbase in codfw |
[production] |
10:08 |
<_joe_> |
running puppet across eqiad varnishes to switch traffic of restbase eqiad => codfw |
[production] |
09:34 |
<volans> |
Copying data from db2009 to db2008 T130098 |
[production] |
09:02 |
<jynus> |
reimporting missing rows from production to labs (expect some lag during the day) |
[production] |
08:59 |
<gehel> |
restarting elasticsearch server elastic1007.eqiad.wmnet |
[production] |
08:44 |
<ori@tin> |
Synchronized php-1.27.0-wmf.16/includes/jobqueue/jobs/RefreshLinksJob.php: Revert: Job queue bankruptcy: force all refreshlinks jobs to be non-recursive (duration: 00m 29s) |
[production] |
08:44 |
<ori@tin> |
Synchronized php-1.27.0-wmf.17/includes/jobqueue/jobs/RefreshLinksJob.php: Revert: Job queue bankruptcy: force all refreshlinks jobs to be non-recursive (duration: 00m 39s) |
[production] |
08:26 |
<ori@tin> |
Synchronized php-1.27.0-wmf.17/includes/jobqueue/jobs/RefreshLinksJob.php: Job queue bankruptcy: force all refreshlinks jobs to be non-recursive (duration: 00m 25s) |
[production] |
08:25 |
<ori@tin> |
Synchronized php-1.27.0-wmf.16/includes/jobqueue/jobs/RefreshLinksJob.php: Job queue bankruptcy: force all refreshlinks jobs to be non-recursive (duration: 00m 25s) |
[production] |
07:48 |
<ori@tin> |
Synchronized php-1.27.0-wmf.16/includes/jobqueue/jobs/RefreshLinksJob.php: Job queue bankruptcy: force all refreshlinks jobs to be non-recursive (duration: 00m 24s) |
[production] |
07:06 |
<ori> |
Short-circuiting RefreshLinksJob::run() to bail if the root job timestamp is older than March 1st |
[production] |
06:46 |
<ori> |
Powercycled elastic1006; unresponsive. |
[production] |
06:11 |
<ebernhardson> |
rebooting elastic1006.eqiad.wmnet for kernel update |
[production] |
03:22 |
<l10nupdate@tin> |
ResourceLoader cache refresh completed at Thu Mar 17 03:22:55 UTC 2016 (duration 9m 33s) |
[production] |
03:14 |
<krenair@tin> |
Synchronized wmf-config/InitialiseSettings-labs.php: https://gerrit.wikimedia.org/r/277452 (labs only change, just keeping file in sync) (duration: 00m 27s) |
[production] |
03:13 |
<mwdeploy@tin> |
sync-l10n completed (1.27.0-wmf.17) (duration: 17m 44s) |
[production] |
02:38 |
<mwdeploy@tin> |
sync-l10n completed (1.27.0-wmf.16) (duration: 17m 36s) |
[production] |
00:57 |
<ebernhardson> |
rebooting elastic1005.eqiad.wmnet for kernel update |
[production] |
00:05 |
<catrope@tin> |
Synchronized wmf-config/InitialiseSettings.php: Enable Flow by default in all talk namespaces on gomwiki (duration: 00m 28s) |
[production] |
2016-03-16
§
|
23:56 |
<ebernhardson> |
rebooting elastic1004.eqiad.wmnet for kernel update |
[production] |
23:47 |
<catrope@tin> |
Finished scap: bswiki namespace changes; MobileFrontend SWAT patches (duration: 05m 04s) |
[production] |
23:42 |
<catrope@tin> |
Started scap: bswiki namespace changes; MobileFrontend SWAT patches |
[production] |
23:15 |
<catrope@tin> |
Synchronized wmf-config/InitialiseSettings.php: Namespace config changes for bswiki (duration: 00m 44s) |
[production] |
22:09 |
<ebernhardson> |
rebooting elastic1003.eqiad.wmnet for linux kernel upgrade |
[production] |
21:47 |
<csteipp> |
deployed patch for T123071 |
[production] |
21:18 |
<gehel> |
restarting elasticsearch server elastic1003.eqiad.wmnet |
[production] |
20:06 |
<mutante> |
mira - git pull in mw-staging |
[production] |
19:58 |
<twentyafterfour@tin> |
rebuilt wikiversions.php and synchronized wikiversions files: group1 wikis to 1.27.0-wmf.17 |
[production] |
19:57 |
<gehel> |
regenerating puppet SSL certificates for elasticsearch eqiad cluster |
[production] |
19:54 |
<mutante> |
rcs1001 - starting redis, disabling puppet (T130147) |
[production] |
19:42 |
<mutante> |
rcs1002 - starting redis-server |
[production] |
19:42 |
<mutante> |
rcs1001 - starting redis-server |
[production] |
19:32 |
<twentyafterfour> |
Deploying 1.27.0-wmf.17 to group1 wikis |
[production] |
19:31 |
<godog> |
restart pybal on lvs1005 T130143 |
[production] |
19:18 |
<legoktm> |
created urlshortcodes table on wikishared db for UrlShortener |
[production] |
19:16 |
<legoktm@tin> |
Synchronized wmf-config/CommonSettings.php: Configure UrlShortener extension in read-only mode (2/2) (duration: 00m 26s) |
[production] |
19:15 |
<legoktm@tin> |
Synchronized wmf-config/InitialiseSettings.php: Configure UrlShortener extension in read-only mode (1/2) (duration: 00m 28s) |
[production] |
19:14 |
<aaron@tin> |
Synchronized php-1.27.0-wmf.17/includes/jobqueue/Job.php: 3da38ce (duration: 00m 37s) |
[production] |
19:12 |
<legoktm@tin> |
Finished scap: Building l10n cache for UrlShortener - T108557 (try #2) (duration: 63m 05s) |
[production] |
19:11 |
<gehel> |
regenerating puppet SSL certificates for elasticsearch codfw cluster |
[production] |
19:08 |
<gehel> |
restarting elasticsearch server elastic1002.eqiad.wmnet |
[production] |
19:02 |
<volans> |
restarted mysql on dbstore2002 and all replicas except x1, still investigating T130128 |
[production] |