|
2018-06-08
§
|
| 17:28 |
<cmjohnson> |
powering flerovium down to move to a different space in the rack |
[production] |
| 17:14 |
<demon@deploy1001> |
Finished deploy [gerrit/gerrit@7324140]: 2.15.2 (duration: 00m 11s) |
[production] |
| 17:14 |
<demon@deploy1001> |
Started deploy [gerrit/gerrit@7324140]: 2.15.2 |
[production] |
| 17:12 |
<no_justification> |
gerrit: taking offline for 2.14 -> 2.15 upgrade |
[production] |
| 15:35 |
<urandom> |
upgrade Cassandra to 3.11.2, restbase1012 - T178905 |
[production] |
| 15:26 |
<marostegui> |
Restart MySQL on codfw sanitarium hosts db2094, db2095 - https://phabricator.wikimedia.org/T196748 |
[production] |
| 15:00 |
<urandom> |
upgrade Cassandra to 3.11.2, restbase1008 - T178905 |
[production] |
| 14:17 |
<urandom> |
upgrade Cassandra to 3.11.2, restbase1011 & restbase1016 - T178905 |
[production] |
| 12:14 |
<twentyafterfour> |
running phabricator public_task_dump script manually to confirm that it's working as expected. |
[production] |
| 11:45 |
<joal> |
Launching manual sqooping of revision and archive table to recover from failure |
[analytics] |
| 09:48 |
<hashar> |
github: deleting https://github.com/wikimedia/mediawiki-extensions-EditOwn | T196433 |
[releng] |
| 09:31 |
<arturo> |
merging https://gerrit.wikimedia.org/r/#/c/436337/ for the eqia1 openstack deployment (labcontrol1003/labcontrol1004) |
[production] |
| 09:19 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-codfw.php: Unify and update sanitarium comments - T190704 (duration: 00m 50s) |
[production] |
| 09:18 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Unify and update sanitarium comments - T190704 (duration: 00m 50s) |
[production] |
| 09:06 |
<akosiaris@deploy1001> |
Started deploy [proton/deploy@97ec4bf]: (no justification provided) |
[production] |
| 08:35 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Repool db1066 after reboot (duration: 00m 50s) |
[production] |
| 08:31 |
<addshore> |
All fixed |
[wikibase-registry] |
| 08:31 |
<addshore> |
docker-compose restart wdqs-frontend |
[wikibase-registry] |
| 08:31 |
<addshore> |
docker-compose restart wdqs-proxy |
[wikibase-registry] |
| 08:30 |
<addshore> |
query service or something seems down |
[wikibase-registry] |
| 08:18 |
<marostegui> |
Stop MySQL and reboot db1066 for intel-microcode install - T194870 |
[production] |
| 08:18 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Depool db1066 for reboot (duration: 00m 50s) |
[production] |
| 07:58 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Repool db1113:3316 after alter table (duration: 00m 51s) |
[production] |
| 07:50 |
<hashar> |
github: deleting archived repo https://github.com/wikimedia/mediawiki-extensions-InterwikiIntegration | T196442 |
[releng] |
| 07:46 |
<hashar> |
nuking integration-slave-jessie-1002 jenkins workspaces |
[releng] |
| 07:46 |
<arturo> |
T196137 more rootspam today, restarting again `prometheus-node-exporter` and force rotating exim4 paniclog in 12 nodes |
[tools] |
| 07:42 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Restore db1091 original weight (duration: 00m 50s) |
[production] |
| 07:25 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Increase weight for db1091 (duration: 00m 50s) |
[production] |
| 07:01 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Increse weight for db1091 (duration: 00m 50s) |
[production] |
| 06:47 |
<gilles@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: T196630 Remove unneeded description from performance survey definition (duration: 00m 52s) |
[production] |
| 06:44 |
<elukey> |
bounce kafka mirror maker main-eqiad-to-main-codfw (kafka200*) due to errors in the logs (also lag metrics not displaying) |
[production] |
| 06:01 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Repool db1091 with low weight after reimage (duration: 00m 50s) |
[production] |
| 05:34 |
<marostegui> |
Deploy sanitarium events on db1124 - T190704 |
[production] |
| 05:23 |
<marostegui> |
Stop MySQL on db1091 for reimage |
[production] |
| 05:22 |
<marostegui> |
Deploy schema change on db1113:3316 - T191316 T192926 T195193 T89737 |
[production] |
| 05:22 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Depool db1113:3316 for alter table (duration: 00m 52s) |
[production] |
| 02:12 |
<mholloway-shell@deploy1001> |
Finished deploy [mobileapps/deploy@0346959]: Update mobileapps to 5ea008c (duration: 00m 42s) |
[production] |
| 02:11 |
<mholloway-shell@deploy1001> |
Started deploy [mobileapps/deploy@0346959]: Update mobileapps to 5ea008c |
[production] |
|
2018-06-07
§
|
| 23:58 |
<demon@deploy1001> |
Finished deploy [gerrit/gerrit@a07d943]: No-op of current deployed version, want to sync repo state (x2) (duration: 00m 14s) |
[production] |
| 23:58 |
<demon@deploy1001> |
Started deploy [gerrit/gerrit@a07d943]: No-op of current deployed version, want to sync repo state (x2) |
[production] |
| 23:57 |
<demon@deploy1001> |
Finished deploy [gerrit/gerrit@a07d943]: No-op of current deployed version, want to sync repo state (duration: 00m 06s) |
[production] |
| 23:57 |
<demon@deploy1001> |
Started deploy [gerrit/gerrit@a07d943]: No-op of current deployed version, want to sync repo state |
[production] |
| 23:13 |
<Krenair> |
armed keyholder on deployment-cumin after reboot, found passphrase at deployment-puppetmaster03:/var/lib/git/labs/private/files/ssh/tin/cumin_rsa.passphrase |
[releng] |
| 21:49 |
<mobrovac@deploy1001> |
Finished deploy [proton/deploy@97ec4bf]: Initial deploy to production - T186748 (duration: 02m 19s) |
[production] |
| 21:47 |
<mobrovac@deploy1001> |
Started deploy [proton/deploy@97ec4bf]: Initial deploy to production - T186748 |
[production] |
| 20:37 |
<herron> |
ircecho restarted |
[production] |
| 20:04 |
<Krenair> |
Enabling puppet and running on shinken-01. "The last Puppet run was at Wed Jun 6 15:58:12 UTC 2018 (1653 minutes ago). Puppet is disabled. silencing shinken during labvirt reboots" |
[shinken] |
| 19:47 |
<urandom> |
rolling Cassandra restart, restbase2005, restbase2006, restbase2012 -- T178905 |
[production] |
| 19:36 |
<XenoRyet> |
updated payments-wiki config from 3197c729ee to b11d120362 |
[production] |
| 19:29 |
<thcipriani@deploy1001> |
rebuilt and synchronized wikiversions files: all wikis to 1.32.0-wmf.7 |
[production] |