|
2016-03-24
§
|
| 19:06 |
<thcipriani@tin> |
rebuilt wikiversions.php and synchronized wikiversions files: all wikis to 1.27.0-wmf.18 |
[production] |
| 18:29 |
<elukey> |
camus and puppet re-enabled on analytics1027 |
[production] |
| 18:18 |
<bblack> |
starting decom/re-role of cp3003-6,cp3011-14 (already depooled) |
[production] |
| 17:57 |
<elukey> |
enabled Hadoop Master Node automatic failover on analytics1001/1002 (this time without fireworks). |
[production] |
| 16:47 |
<urandom> |
Reactivating restbase2004.codfw.wmnet : T130254] |
[production] |
| 16:43 |
<filippo@tin> |
Synchronized wmf-config/filebackend-production.php: move codfw swift back to async replication T129089 (duration: 00m 28s) |
[production] |
| 16:41 |
<ebernhardson@tin> |
Synchronized wmf-config/CirrusSearch-labs.php: T130219 CirrusSearch labs configured to use HTTPS connection pool (duration: 00m 30s) |
[production] |
| 16:31 |
<godog> |
force puppet run on upload caches in eqiad for https://gerrit.wikimedia.org/r/#/c/279358/ |
[production] |
| 16:25 |
<bblack> |
depooling cp3003-6,cp3011-14 from esams text varnish-be over the next ~70 mins - prep for last steps of T125485 |
[production] |
| 16:22 |
<demon@tin> |
Synchronized README: no-op, doing co-master sync (duration: 00m 38s) |
[production] |
| 16:17 |
<godog> |
force puppet run on upload caches in codfw/eqiad for https://gerrit.wikimedia.org/r/#/c/279357/ |
[production] |
| 16:15 |
<elukey> |
disabled camus and puppet on analytics1027 |
[production] |
| 16:10 |
<godog> |
force puppet run on upload caches in eqiad for https://gerrit.wikimedia.org/r/#/c/279356/ |
[production] |
| 15:46 |
<bblack> |
starting depool/downtime/re-role process to decom cp3019-22 |
[production] |
| 15:43 |
<bblack> |
restarted pybal on lvs300x (persistent failure to load new etcd data...) |
[production] |
| 15:38 |
<moritzm> |
upgrading xenon, cerium, praseodymium to Linux 4.4 |
[production] |
| 15:09 |
<bblack> |
downtime/re-role of cp3007-10 starting - T125485 |
[production] |
| 14:21 |
<bblack> |
restarted ganglia-monitor-aggregator instances on hooft.esams |
[production] |
| 14:17 |
<ema> |
puppet re-enabled on all cache nodes T124279 |
[production] |
| 13:29 |
<ema> |
testing vcl v4 merge, puppet disabled on all cache nodes T124279 |
[production] |
| 13:22 |
<jynus@tin> |
Synchronized wmf-config/db-eqiad.php: Repool db1027; pool db1078 with low weight (duration: 00m 42s) |
[production] |
| 13:17 |
<elukey> |
puppet re-enabled on analytics1027 |
[production] |
| 11:20 |
<hashar> |
Restarted Jenkins, jobs got blocked notifying to IRC |
[production] |
| 11:10 |
<gehel> |
running puppet-merge manually on strontium to correct unmerged changes. |
[production] |
| 10:18 |
<gehel> |
activating SSL certificate check on elasticsearch - T130366 |
[production] |
| 10:07 |
<moritzm> |
installing exim upgrades on non-mail hubs in production |
[production] |
| 09:56 |
<elukey> |
puppet disabled on analitycs1027 to disable Camus |
[production] |
| 09:50 |
<elukey> |
puppet disabled on analytics1001/1002 (HDFS namenodes) as pre-step before enabling automatic failover |
[production] |
| 09:30 |
<jynus> |
stopping slave on db2018 |
[production] |
| 09:18 |
<jynus> |
stopping db1027 for upgrade and cloning to db1078 |
[production] |
| 08:57 |
<jynus@tin> |
Synchronized wmf-config/db-eqiad.php: Repool db1044; Pool db1077; Depool db1027 (duration: 00m 31s) |
[production] |
| 08:56 |
<jynus@tin> |
Synchronized wmf-config/db-codfw.php: Add db1077 and db1078 (duration: 00m 28s) |
[production] |
| 05:05 |
<Krinkle> |
mwscript deleteEqualMessages.php --wiki zhwikiquote (T45917 , P1988) |
[production] |
| 05:05 |
<Krinkle> |
mwscript deleteEqualMessages.php --wiki zhwiktionary |
[production] |
| 05:03 |
<Krinkle> |
mwscript deleteEqualMessages.php --wiki zhwiki |
[production] |
| 05:03 |
<Krinkle> |
mwscript deleteEqualMessages.php --wiki zhwikibooks |
[production] |
| 04:58 |
<krinkle@tin> |
Synchronized wmf-config/InitialiseSettings.php: 3f2943e (duration: 00m 26s) |
[production] |
| 04:54 |
<krinkle@tin> |
Synchronized wmf-config/CommonSettings.php: 3f2943e (duration: 00m 41s) |
[production] |
| 04:43 |
<krinkle@tin> |
Synchronized wmf-config/CommonSettings-labs.php: (no message) (duration: 00m 35s) |
[production] |
| 03:02 |
<l10nupdate@tin> |
ResourceLoader cache refresh completed at Thu Mar 24 03:02:09 UTC 2016 (duration 8m 52s) |
[production] |
| 02:53 |
<mwdeploy@tin> |
sync-l10n completed (1.27.0-wmf.18) (duration: 09m 59s) |
[production] |
| 02:27 |
<mwdeploy@tin> |
sync-l10n completed (1.27.0-wmf.17) (duration: 11m 07s) |
[production] |
| 00:47 |
<twentyafterfour> |
phab upgrade/maintenance finished |
[production] |
| 00:47 |
<ori@tin> |
Synchronized php-1.27.0-wmf.18/includes/media/DjVu.php: Idbd11637a8: Request-local caching of DjVu dimensions (duration: 00m 39s) |
[production] |
| 00:13 |
<catrope@tin> |
Synchronized php-1.27.0-wmf.18/extensions/Echo/modules/ooui/mw.echo.ui.FooterNoticeWidget.js: touch (duration: 00m 26s) |
[production] |
| 00:06 |
<catrope@tin> |
Finished scap: Oops, Echo had i18n changes (duration: 26m 50s) |
[production] |
| 00:02 |
<moritzm> |
rebooting iridium (phabricator host) for kernel upgrade |
[production] |