|
2013-10-22
§
|
| 10:09 |
<apergos> |
powercycled sq80, hung on mgmt console |
[production] |
| 09:42 |
<hashar> |
Zuul: no more voting on code-review label {{bug|55757}} {{gerrit|91157}}. Will make the database happier I guess |
[production] |
| 09:30 |
<springle> |
synchronized wmf-config/db-eqiad.php 'repool db1018' |
[production] |
| 03:46 |
<springle> |
synchronized wmf-config/db-eqiad.php 'repool db1007, warm up db1041' |
[production] |
| 03:06 |
<LocalisationUpdate> |
ResourceLoader cache refresh completed at Tue Oct 22 03:06:46 UTC 2013 |
[production] |
| 03:00 |
<springle> |
upgrading db1017 to precise |
[production] |
| 02:39 |
<LocalisationUpdate> |
completed (1.22wmf21) at Tue Oct 22 02:39:32 UTC 2013 |
[production] |
| 02:25 |
<LocalisationUpdate> |
completed (1.22wmf22) at Tue Oct 22 02:25:12 UTC 2013 |
[production] |
| 01:41 |
<springle> |
start xtrabackup clone db1007 to db1041 |
[production] |
| 01:26 |
<springle> |
start xtrabackup clone db1018 to db1034 |
[production] |
| 01:07 |
<springle> |
upgrading db1034 & db1041 to precise |
[production] |
| 00:53 |
<springle> |
synchronized wmf-config/db-eqiad.php 'depool db1007 while cloning' |
[production] |
| 00:46 |
<awight> |
crm schema change: wmf_civicrm 6008->6009 |
[production] |
| 00:40 |
<springle> |
synchronized wmf-config/db-eqiad.php 'depool db1018 while cloning' |
[production] |
| 00:36 |
<awight> |
manual schema change on crm: update system set schema_version=6008 where name='wmf_civicrm'; |
[production] |
| 00:24 |
<awight> |
fundraising-crm updated from 9247de6335d74d32447944eef467960edc74ae84 to 41b621ab759fbafc9dcc9cf8323d1bdd973dfb72 |
[production] |
|
2013-10-21
§
|
| 23:22 |
<mwalker> |
synchronized php-1.22wmf21/extensions/CentralNotice/ 'Reverting earlier change' |
[production] |
| 23:21 |
<mwalker> |
synchronized php-1.22wmf22/extensions/CentralNotice/ 'Reverting earlier change' |
[production] |
| 23:17 |
<mwalker> |
CentralNotice deploy had an issue where we are now pushing ALL traffic to the mobile site for CN -- ganglia reports a spike and I'm now reverting the change |
[production] |
| 23:09 |
<^demon|lunch> |
elastic: index created for bnwiki, running force indexing in 4 processes on terbium |
[production] |
| 23:07 |
<demon> |
synchronized wmf-config/InitialiseSettings.php 'bnwiki gets cirrus as secondary' |
[production] |
| 23:05 |
<mwalker> |
synchronized php-1.22wmf22/extensions/CentralNotice/ 'Updating CentralNotice to master' |
[production] |
| 23:05 |
<mwalker> |
synchronized php-1.22wmf21/extensions/CentralNotice/ 'Updating CentralNotice to master' |
[production] |
| 22:49 |
<paravoid> |
powercycling ms-be1004, locked up(?) |
[production] |
| 22:13 |
<maxsem> |
synchronized php-1.22wmf22/extensions/MobileFrontend/ 'https://gerrit.wikimedia.org/r/91065' |
[production] |
| 22:04 |
<AaronSchulz> |
Running syncFilebackend on all wikis (should reduce sync errors; a few a popping up in the logs) |
[production] |
| 21:34 |
<aaron> |
synchronized wmf-config/filebackend.php 'Added eqiad swift to multiwrite backends' |
[production] |
| 21:06 |
<cmjohnson1> |
morebots are you working? |
[production] |
| 21:04 |
<cmjohnson1> |
dns update |
[production] |
| 20:05 |
<cmjohnson1> |
dns update |
[production] |
| 19:38 |
<cmjohnson1> |
dns update |
[production] |
| 18:40 |
<reedy> |
synchronized wmf-config/interwiki.cdb 'Updating interwiki cache' |
[production] |
| 18:34 |
<reedy> |
rebuilt wikiversions.cdb and synchronized wikiversions files: Non wikipedias to 1.22wmf22 |
[production] |
| 18:29 |
<reedy> |
synchronized wmf-config/InitialiseSettings.php |
[production] |
| 17:44 |
<cmjohnson1> |
reclaiming cp1021-36, cp1041-42 per RT5981 |
[production] |
| 17:24 |
<Reedy> |
The @Wikimedia Operations team is seeking proposals for a new datacenter in the midwestern/western US: https://blog.wikimedia.org/2013/10/21/rfp-new-datacenter-continental-us/ |
[production] |
| 16:30 |
<ori-l> |
ran sync-apache for I1113b9594 & Ie73ce6213; verified on mw60; gracefuling. |
[production] |
| 16:26 |
<demon> |
synchronized wmf-config/CirrusSearch-production.php 'LVS for cirrus on all wikis' |
[production] |
| 16:11 |
<demon> |
synchronized wmf-config/CirrusSearch-production.php 'LVS for cirrus on test2wiki' |
[production] |
| 16:07 |
<RobH> |
updated akismet blog plugin |
[production] |
| 16:03 |
<RobH> |
tweaked permissions of the blog's w3 caching plugin to actually make it work again |
[production] |
| 11:47 |
<mark> |
Shutdown csw2-esams:xe-2/1/1 (1 DF leg) |
[production] |
| 11:31 |
<hashar> |
Gerrit replication have been broken since Oct 19th roughly 20:50 UTC. {{bug|55948}} |
[production] |
| 10:43 |
<hashar> |
Gerrit replication is broken, side effects: git.wm.o shows outdated trees and Jenkins might be missing some commits |
[production] |
| 09:16 |
<reedy> |
synchronized wmf-config/InitialiseSettings.php |
[production] |
| 09:16 |
<reedy> |
synchronized docroot/bits/apple-touch/ |
[production] |
| 09:03 |
<hashar> |
restarting Zuul |
[production] |
| 09:02 |
<hashar> |
Jenkins restarted / upgraded |
[production] |
| 08:59 |
<hashar> |
rerestarting Jenkins. |
[production] |
| 08:51 |
<akosiaris> |
forced verified +2 on gerrit 90874 since jenkins is being upgraded |
[production] |