|
2017-12-07
§
|
| 10:50 |
<elukey> |
powercycle analytics1003 - no serial console, ssh stuck in System is booting up. See pam_nologin(8) |
[production] |
| 10:28 |
<_joe_> |
depooling mw1283 for further investigation |
[production] |
| 10:14 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Fully pool db1098:3316 db1098:3317 - T178359 (duration: 00m 51s) |
[production] |
| 10:12 |
<elukey> |
reboot analytics1003 for kernel+jvm updates - T179943 |
[production] |
| 10:01 |
<gehel> |
upgrade of ELK stack on logstash100* completed - Kibana was unavailable for longer than expected - T178412 |
[production] |
| 09:48 |
<hashar> |
CI: removed Wikidata from configuration, replaced by Wikibase. wmf/* and REL branches are going to be broken though | https://gerrit.wikimedia.org/r/395704 | T181838 |
[production] |
| 09:46 |
<akosiaris> |
silence ganeti1006 on icinga T181121 |
[production] |
| 09:36 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Increase traffic for db1098:3316 db1098:3317 - T178359 (duration: 00m 52s) |
[production] |
| 09:28 |
<marostegui> |
Upgrade MySQL and kernel on db1074 |
[production] |
| 09:22 |
<gehel@puppetmaster1001> |
conftool action : set/pooled=yes; selector: name=logstash1009.eqiad.wmnet |
[production] |
| 09:19 |
<gehel@puppetmaster1001> |
conftool action : set/pooled=no; selector: name=logstash1009.eqiad.wmnet |
[production] |
| 09:18 |
<gehel@puppetmaster1001> |
conftool action : set/pooled=yes; selector: name=logstash1008.eqiad.wmnet |
[production] |
| 09:14 |
<gehel@puppetmaster1001> |
conftool action : set/pooled=no; selector: name=logstash1008.eqiad.wmnet |
[production] |
| 09:13 |
<gehel@puppetmaster1001> |
conftool action : set/pooled=yes; selector: name=logstash1007.eqiad.wmnet |
[production] |
| 09:08 |
<gehel@puppetmaster1001> |
conftool action : set/pooled=no; selector: name=logstash1007.eqiad.wmnet |
[production] |
| 09:05 |
<gehel@tin> |
Finished deploy [logstash/plugins@b13d2fa]: (no justification provided) (duration: 00m 02s) |
[production] |
| 09:05 |
<gehel@tin> |
Started deploy [logstash/plugins@b13d2fa]: (no justification provided) |
[production] |
| 09:00 |
<gehel> |
upgrading ELK stack on logstash100* - some log messages might be lost during the upgrade - T178412 |
[production] |
| 08:51 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Increase traffic for db1098:3316 db1098:3317 - T178359 (duration: 00m 48s) |
[production] |
| 08:32 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Increase traffic for db1098:3316 db1098:3317 - T178359 (duration: 00m 48s) |
[production] |
| 08:28 |
<elukey> |
install prometheus-druid-exporter 0.5 on druid* |
[production] |
| 08:26 |
<elukey> |
upload prometheus-druid-exporter 0.5-1 to jessie/stretch-wikimedia |
[production] |
| 07:19 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Slowly pool db1098:3316 db1098:3317 - T178359 (duration: 00m 47s) |
[production] |
| 07:18 |
<marostegui@tin> |
Synchronized wmf-config/db-codfw.php: Slowly pool db1098:3316 db1098:3317 - T178359 (duration: 00m 48s) |
[production] |
| 06:45 |
<marostegui> |
Stop replication on db1099:3311 to reimport: change_tag, tag_summary, user and watchlist tables and recompress again - T178359 |
[production] |
| 06:40 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Depool db1074 - T174569 (duration: 00m 48s) |
[production] |
| 06:40 |
<marostegui> |
Deploy schema change on db1074 (s2) - T174569 |
[production] |
| 03:00 |
<catrope@tin> |
Synchronized php-1.31.0-wmf.11/resources/src/mediawiki.rcfilters/: T182268 (duration: 00m 57s) |
[production] |
| 02:29 |
<l10nupdate@tin> |
scap sync-l10n completed (1.31.0-wmf.10) (duration: 08m 57s) |
[production] |
|
2017-12-06
§
|
| 23:45 |
<eileen> |
update CiviCRM from 85263f625763a454b2664062154c8b791e5a6838 to 2db9c761884c1c48f4e1dd41be416e029b23b481 (improve org merges) |
[production] |
| 23:12 |
<ppchelko@tin> |
Started restart [changeprop/deploy@065a06e]: (no justification provided) |
[production] |
| 23:04 |
<eileen> |
update process-control to b1cd5150e8ef2bf9e4212 (reenable bounce processing, enable dedupe on orgs) |
[production] |
| 22:58 |
<ppchelko@tin> |
Finished deploy [cpjobqueue/deploy@761524f]: Separate delay and totaldelay metrics T182216 (duration: 00m 31s) |
[production] |
| 22:57 |
<ppchelko@tin> |
Started deploy [cpjobqueue/deploy@761524f]: Separate delay and totaldelay metrics T182216 |
[production] |
| 22:55 |
<mutante> |
stat1003 - re-enabled puppet after putting role::spare on it (T175150) |
[production] |
| 22:48 |
<mutante> |
stat1003 - this host was kind of invisible (not in site, not in icinga) but still up, re-enabling puppet after re-adding it to site |
[production] |
| 22:03 |
<demon@tin> |
Synchronized php-1.31.0-wmf.11/extensions/WikidataPageBanner/includes/WikidataPageBanner.hooks.php: unbreak (duration: 00m 48s) |
[production] |
| 22:00 |
<demon@tin> |
rebuilt wikiversions.php and synchronized wikiversions files: group1 to wmf.11, again |
[production] |
| 21:51 |
<hoo@tin> |
Synchronized php-1.31.0-wmf.10/extensions/Wikibase/lib/includes/Changes/EntityChange.php: Make EntityChange truly forward compatible with compact diffs (T182243) (duration: 00m 48s) |
[production] |
| 21:43 |
<arlolra> |
Updated Parsoid to 01c1fc3 (T178253, T61840, T180930, T179259) |
[production] |
| 21:34 |
<arlolra@tin> |
Finished deploy [parsoid/deploy@dfcc622]: Updating Parsoid to 01c1fc3 (duration: 09m 45s) |
[production] |
| 21:24 |
<arlolra@tin> |
Started deploy [parsoid/deploy@dfcc622]: Updating Parsoid to 01c1fc3 |
[production] |
| 20:45 |
<marostegui> |
Add 400G to labsdb1003 /srv partition |
[production] |
| 20:15 |
<hoo> |
Ran "scap pull" on snapshot1001 after T177486 related tests |
[production] |
| 20:10 |
<demon@tin> |
rebuilt wikiversions.php and synchronized wikiversions files: partial rollback -- wikidata errors |
[production] |
| 20:07 |
<demon@tin> |
rebuilt wikiversions.php and synchronized wikiversions files: group1 to wmf.11 |
[production] |
| 20:05 |
<demon@tin> |
Synchronized php: symlink bump (duration: 00m 47s) |
[production] |
| 19:32 |
<hoo@tin> |
Synchronized php-1.31.0-wmf.11/extensions/Wikibase/repo/maintenance/dispatchChanges.php: dispatch: track how long client selecting takes (duration: 00m 48s) |
[production] |
| 19:24 |
<ppchelko@tin> |
Finished deploy [cpjobqueue/deploy@8c66189]: Deduplicate based on event sha1 as well as on id, take 2 (duration: 00m 38s) |
[production] |
| 19:24 |
<ppchelko@tin> |
Started deploy [cpjobqueue/deploy@8c66189]: Deduplicate based on event sha1 as well as on id, take 2 |
[production] |