|
2016-07-07
ยง
|
| 16:24 |
<gehel> |
starting elasticsearch and kibana upgrade on logstash cluster (T136001) |
[production] |
| 16:21 |
<awight> |
Taking Fundraising campaigns down for maintenance |
[production] |
| 15:51 |
<elukey> |
add mw1261 back into service |
[production] |
| 15:44 |
<bd808> |
Dropped logstash indices older than logstash-2016.07.01 in preparation for elasticsearch upgrade |
[production] |
| 15:25 |
<thcipriani@tin> |
Synchronized php-1.28.0-wmf.9/extensions/Echo/modules/controller/mw.echo.Controller.js: SWAT: [[gerrit:297792|Correct section (alert/message/all)]] (duration: 00m 25s) |
[production] |
| 15:13 |
<_joe_> |
uploaded new HHVM package for jessie |
[production] |
| 15:11 |
<thcipriani@tin> |
Synchronized dblists/clldefault.dblist: SWAT: [[gerrit:297349|Deploy Compact Language Links as default (Stage 4) (T136677)]] PART II (duration: 00m 34s) |
[production] |
| 15:11 |
<thcipriani@tin> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:297349|Deploy Compact Language Links as default (Stage 4) (T136677)]] PART I (duration: 00m 55s) |
[production] |
| 14:43 |
<elukey@palladium> |
conftool action : set/pooled=yes; selector: aqs1001.eqiad.wmnet |
[production] |
| 14:38 |
<elukey@palladium> |
conftool action : set/pooled=no; selector: aqs1001.eqiad.wmnet |
[production] |
| 14:37 |
<elukey> |
depool aqs1001 for nodejs upgrade |
[production] |
| 13:10 |
<hashar> |
deleting integration-slave-trusty-1024 and integration-slave-trusty-1025 to free up some RAM. We have enough permanent Trusty slaves. T139535 |
[releng] |
| 12:45 |
<yuvipanda> |
start deployment of k8s 1.3.0wmf4 for T139259 |
[tools] |
| 12:05 |
<elukey> |
depooling mw1261 from service (T73487) |
[production] |
| 11:34 |
<jynus> |
breaking m3 replication on db1048 (depooled) to check icinga alert changes |
[production] |
| 10:44 |
<jynus> |
disabling all mysql lag alerts cross-fleet T122457 |
[production] |
| 10:29 |
<akosiaris> |
reboot fermium.wikimedia.org hassium.eqiad.wmnet install1001.wikimedia.org krypton.eqiad.wmnet meitnerium.wikimedia.org mendelevium.eqiad.wmnet T134242 |
[production] |
| 10:25 |
<akosiaris> |
reboot mx1001, planet1001, rutherfordium, seaborgium, ununpentium T134242 |
[production] |
| 10:22 |
<akosiaris> |
reboot dubnium T134242 |
[production] |
| 10:22 |
<akosiaris> |
reboot bromine T134242 |
[production] |
| 10:13 |
<akosiaris> |
reboot bohrium T134242 |
[production] |
| 10:11 |
<elukey> |
pooling mw1261 back to service with Apache mod-proxy-fcgi set to trace8 (T73487) |
[production] |
| 10:07 |
<akosiaris> |
reboot etherpad1001.eqiad.wmnet, kernel upgrade and qemu upgrade, T134242 |
[production] |
| 10:04 |
<gehel> |
rolling restart of elasticsearch cluster eqiad completed (T138811) |
[production] |
| 09:57 |
<paladox> |
zuul, gerrit and jenkins are all setup now (Thanks hashar for helping me) |
[phabricator] |
| 08:51 |
<_joe_> |
removing all old servers from the appservers pool but the canaries (T139353) |
[production] |
| 08:47 |
<jynus@tin> |
Synchronized wmf-config/db-eqiad.php: Failover s4 recentchanges to db1056 (duration: 00m 38s) |
[production] |
| 08:33 |
<hoo> |
Updated Wikidata's property suggester with data from Monday's json dump and removed the external identifiers as a workaround for T132839 |
[production] |
| 07:56 |
<gehel> |
rolling restart of elasticsearch cluster codfw completed (T138811) |
[production] |
| 07:16 |
<legoktm> |
mysql:wikiadmin@db1041 [centralauth]> delete from localuser where lu_name ="Philippe" and lu_wiki ="scnwiki"; |
[production] |
| 07:11 |
<Matthew_> |
Restarted wm-bot, RSS feed error again |
[bots] |
| 04:37 |
<eileen> |
Update CiviCRM from dd24368a897fd78752178ee253e7a890dd57db41 to bdf2afd417b70332c9542fd3ee4f14cb4e6f93cc |
[production] |
| 03:12 |
<eileen> |
CiviCRM upgrade from 5f8f7c3236e6bf12c52deea07093fbca165ef4a7 to dd24368a897fd78752178ee253e7a890dd57db41 |
[production] |
| 03:07 |
<chasemp> |
silence labvirt1011 flapping for 24h, we have a task, and we are attempting to move vms we can |
[production] |
| 02:48 |
<l10nupdate@tin> |
ResourceLoader cache refresh completed at Thu Jul 7 02:48:55 UTC 2016 (duration 6m 44s) |
[production] |
| 02:43 |
<MaxSem> |
started redis-server on deployment-stream |
[releng] |
| 02:42 |
<mwdeploy@tin> |
scap sync-l10n completed (1.28.0-wmf.9) (duration: 07m 07s) |
[production] |
| 02:26 |
<mwdeploy@tin> |
scap sync-l10n completed (1.28.0-wmf.8) (duration: 09m 58s) |
[production] |
| 02:14 |
<chasemp> |
reboot labvirt1011 |
[production] |
| 01:53 |
<chasemp> |
start nova-compute again as it seems ot have been killed on labvirt1011 which is acting weird |
[production] |
| 01:24 |
<andrewbogott> |
rebooting labvirt1011 (because it is acting crazy) |
[production] |
| 01:14 |
<bd808> |
Restarted logstash on deployment-logstash2 |
[releng] |
| 01:12 |
<MaxSem> |
Leaving my hacks for the night to collect data, if needed revert with cd /srv/mediawiki-staging/php-master/vendor && sudo git reset --hard HEAD && sudo chown -hR jenkins-deploy:wikidev . |
[releng] |
| 00:50 |
<eileen> |
updating CiviCRM from 54e168db2fddc6a9a07036323e01a27dd64333cf to 5f8f7c3236e6bf12c52deea07093fbca165ef4a7 |
[production] |
| 00:50 |
<bd808> |
Rebooting deployment-logstash3.eqiad.wmflabs; console full of hung process messages from kernel |
[releng] |
| 00:27 |
<MaxSem> |
Initialized ORES on all wikis where it's enabled, was causing job failures |
[releng] |
| 00:23 |
<eileen> |
Updated CiviCRM from bb9bf136dc0fa82d5d07ebeb33d696e54672b2d6 to 54e168db2fddc6a9a07036323e01a27dd64333cf |
[production] |
| 00:13 |
<MaxSem> |
Debugging a fatal in betalabs, might cause syncs to fail |
[releng] |
| 00:06 |
<legoktm@tin> |
Synchronized php-1.28.0-wmf.8/extensions/CentralAuth/: Make LocalRename jobs run sequentially - T137973 (for real this time) (duration: 00m 30s) |
[production] |
| 00:05 |
<legoktm@tin> |
Synchronized php-1.28.0-wmf.9/extensions/CentralAuth/: Make LocalRename jobs run sequentially - T137973 (duration: 00m 30s) |
[production] |