|
2018-06-08
§
|
| 22:41 |
<no_justification> |
gerrit: restarting |
[production] |
| 20:18 |
<Hauskatze> |
Updated stewardbots to 5ec794e |
[tools.stewardbots] |
| 19:36 |
<urandom> |
upgrade Cassandra to 3.11.2, restbase1015 & restbase1018 - T178905 |
[production] |
| 19:02 |
<framawiki> |
"qstat" says that job 3838834 maj_articles_recents is live still 04/13/2018 (?) but logs indicate that it's currently stuck, so "qdel 3838834", cron has restarted it and it's editing again |
[tools.totoazero] |
| 18:49 |
<urandom> |
upgrade Cassandra to 3.11.2, restbase1009 & restbase1014 - T178905 |
[production] |
| 18:05 |
<legoktm> |
updating docker files for https://gerrit.wikimedia.org/r/438152 |
[releng] |
| 18:02 |
<urandom> |
upgrade Cassandra to 3.11.2, restbase1017 & restbase1013 - T178905 |
[production] |
| 17:33 |
<no_justification> |
gerrit: up mostly, but will see some errors about "wip" label for a bit until reindexing completes. |
[production] |
| 17:28 |
<cmjohnson> |
powering flerovium down to move to a different space in the rack |
[production] |
| 17:14 |
<demon@deploy1001> |
Finished deploy [gerrit/gerrit@7324140]: 2.15.2 (duration: 00m 11s) |
[production] |
| 17:14 |
<demon@deploy1001> |
Started deploy [gerrit/gerrit@7324140]: 2.15.2 |
[production] |
| 17:12 |
<no_justification> |
gerrit: taking offline for 2.14 -> 2.15 upgrade |
[production] |
| 15:35 |
<urandom> |
upgrade Cassandra to 3.11.2, restbase1012 - T178905 |
[production] |
| 15:26 |
<marostegui> |
Restart MySQL on codfw sanitarium hosts db2094, db2095 - https://phabricator.wikimedia.org/T196748 |
[production] |
| 15:00 |
<urandom> |
upgrade Cassandra to 3.11.2, restbase1008 - T178905 |
[production] |
| 14:17 |
<urandom> |
upgrade Cassandra to 3.11.2, restbase1011 & restbase1016 - T178905 |
[production] |
| 12:14 |
<twentyafterfour> |
running phabricator public_task_dump script manually to confirm that it's working as expected. |
[production] |
| 11:45 |
<joal> |
Launching manual sqooping of revision and archive table to recover from failure |
[analytics] |
| 09:48 |
<hashar> |
github: deleting https://github.com/wikimedia/mediawiki-extensions-EditOwn | T196433 |
[releng] |
| 09:31 |
<arturo> |
merging https://gerrit.wikimedia.org/r/#/c/436337/ for the eqia1 openstack deployment (labcontrol1003/labcontrol1004) |
[production] |
| 09:19 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-codfw.php: Unify and update sanitarium comments - T190704 (duration: 00m 50s) |
[production] |
| 09:18 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Unify and update sanitarium comments - T190704 (duration: 00m 50s) |
[production] |
| 09:06 |
<akosiaris@deploy1001> |
Started deploy [proton/deploy@97ec4bf]: (no justification provided) |
[production] |
| 08:35 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Repool db1066 after reboot (duration: 00m 50s) |
[production] |
| 08:31 |
<addshore> |
All fixed |
[wikibase-registry] |
| 08:31 |
<addshore> |
docker-compose restart wdqs-frontend |
[wikibase-registry] |
| 08:31 |
<addshore> |
docker-compose restart wdqs-proxy |
[wikibase-registry] |
| 08:30 |
<addshore> |
query service or something seems down |
[wikibase-registry] |
| 08:18 |
<marostegui> |
Stop MySQL and reboot db1066 for intel-microcode install - T194870 |
[production] |
| 08:18 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Depool db1066 for reboot (duration: 00m 50s) |
[production] |
| 07:58 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Repool db1113:3316 after alter table (duration: 00m 51s) |
[production] |
| 07:50 |
<hashar> |
github: deleting archived repo https://github.com/wikimedia/mediawiki-extensions-InterwikiIntegration | T196442 |
[releng] |
| 07:46 |
<hashar> |
nuking integration-slave-jessie-1002 jenkins workspaces |
[releng] |
| 07:46 |
<arturo> |
T196137 more rootspam today, restarting again `prometheus-node-exporter` and force rotating exim4 paniclog in 12 nodes |
[tools] |
| 07:42 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Restore db1091 original weight (duration: 00m 50s) |
[production] |
| 07:25 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Increase weight for db1091 (duration: 00m 50s) |
[production] |
| 07:01 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Increse weight for db1091 (duration: 00m 50s) |
[production] |
| 06:47 |
<gilles@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: T196630 Remove unneeded description from performance survey definition (duration: 00m 52s) |
[production] |
| 06:44 |
<elukey> |
bounce kafka mirror maker main-eqiad-to-main-codfw (kafka200*) due to errors in the logs (also lag metrics not displaying) |
[production] |
| 06:01 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Repool db1091 with low weight after reimage (duration: 00m 50s) |
[production] |
| 05:34 |
<marostegui> |
Deploy sanitarium events on db1124 - T190704 |
[production] |
| 05:23 |
<marostegui> |
Stop MySQL on db1091 for reimage |
[production] |
| 05:22 |
<marostegui> |
Deploy schema change on db1113:3316 - T191316 T192926 T195193 T89737 |
[production] |
| 05:22 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Depool db1113:3316 for alter table (duration: 00m 52s) |
[production] |
| 02:12 |
<mholloway-shell@deploy1001> |
Finished deploy [mobileapps/deploy@0346959]: Update mobileapps to 5ea008c (duration: 00m 42s) |
[production] |
| 02:11 |
<mholloway-shell@deploy1001> |
Started deploy [mobileapps/deploy@0346959]: Update mobileapps to 5ea008c |
[production] |