|
2017-11-22
§
|
| 10:54 |
<akosiaris> |
gnt-node migrate -f ganeti1005. T181121 |
[production] |
| 10:51 |
<marostegui> |
Drop index from ores_classification on s5 - T180045 |
[production] |
| 10:44 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Pool db1051 and db1063 in vslow service for s5 to warm them up for the s8 split - T177208 (duration: 00m 45s) |
[production] |
| 10:23 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Add db1101 to s5 and s7 as recentchanges multi-instance slave - T178359 (duration: 00m 45s) |
[production] |
| 10:04 |
<moritzm> |
running "scap pull" on mw1191, it's depooled and marked as "inactive", but health checks are triggering db errors |
[production] |
| 09:35 |
<bblack> |
cr[12]-ulsfo - switch static fallback LVS routes from lvs400[12] to lvs400[56] |
[production] |
| 09:27 |
<bblack> |
lvs@ulsfo - done switching primaries (host MED config) - lvs400[56] now primary for text/upload traffic |
[production] |
| 09:11 |
<akosiaris@tin> |
Finished deploy [parsoid/deploy@b150764]: T180211 (duration: 05m 05s) |
[production] |
| 09:08 |
<bblack> |
puppet disabled on lvs400[1256] for switching primaries |
[production] |
| 09:06 |
<akosiaris@tin> |
Started deploy [parsoid/deploy@b150764]: T180211 |
[production] |
| 09:04 |
<akosiaris@puppetmaster1001> |
conftool action : set/pooled=yes; selector: name=wtp2017.codfw.wmnet |
[production] |
| 09:00 |
<bblack> |
lvs4005 - reboot to clear experimental stuff |
[production] |
| 08:16 |
<bblack> |
backend restart on cp4024 (upload@ulsfo) - mailbox lag |
[production] |
| 07:56 |
<marostegui> |
Drop index from ores_classification on s3 - T180045 |
[production] |
| 07:50 |
<marostegui> |
Drop index from ores_classification on s6 - T180045 |
[production] |
| 07:48 |
<marostegui> |
Drop index from ores_classification on s7 - T180045 |
[production] |
| 07:29 |
<_joe_> |
stopping the additional workers for htmlCacheUpdate (commons and ruwiki), adding one additional runner for refreshLinks on ruwiki |
[production] |
| 06:43 |
<marostegui> |
Stop MySQL on db1063 and db1051 (which is going to be recloned) - T177208 |
[production] |
| 06:42 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Move db1051 from s1 to s5 - T177208 (duration: 00m 53s) |
[production] |
| 06:28 |
<ejegg> |
updated payments-wiki from f871160c83d5be95bf25587aaf8bb87b8b56a809 to 1ca91b1b029161457c86f4f403be0ac78e715d79 |
[production] |
| 03:36 |
<mutante> |
powercycled mw2251 which had gone down without further comment |
[production] |
| 02:35 |
<l10nupdate@tin> |
scap sync-l10n completed (1.31.0-wmf.8) (duration: 11m 34s) |
[production] |
| 01:09 |
<hoo> |
Cleaned out remaining T180934 related log blow up on snapshot1007 (dumpwikidatajson-wikidata-20171120-all-0.log) |
[production] |
| 00:43 |
<mutante> |
gerrit restarting service to apply config change |
[production] |
| 00:40 |
<mutante> |
gerrit - re-enabling puppet to apply logstash change on cobalt, gerrit restart incoming (T141324) |
[production] |
| 00:39 |
<maxsem@tin> |
Synchronized php-1.31.0-wmf.8/extensions/CentralNotice/: https://gerrit.wikimedia.org/r/#/c/392754/ (duration: 00m 47s) |
[production] |
| 00:11 |
<maxsem@tin> |
Synchronized php-1.31.0-wmf.8/extensions/InputBox/: https://gerrit.wikimedia.org/r/#/c/392745/ (duration: 00m 45s) |
[production] |
| 00:10 |
<maxsem@tin> |
Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/392576/ (duration: 00m 46s) |
[production] |
| 00:07 |
<bblack@neodymium> |
conftool action : set/pooled=yes; selector: name=nescio.wikimedia.org |
[production] |
| 00:04 |
<bblack@neodymium> |
conftool action : set/pooled=no; selector: name=nescio.wikimedia.org |
[production] |
| 00:02 |
<bblack@neodymium> |
conftool action : set/pooled=yes; selector: name=maerlant.wikimedia.org |
[production] |
| 00:01 |
<bblack@neodymium> |
conftool action : set/pooled=no; selector: name=maerlant.wikimedia.org |
[production] |
| 00:01 |
<bblack@neodymium> |
conftool action : set/pooled=yes; selector: name=maerlant.wikimedia.org |
[production] |
| 00:00 |
<bblack@neodymium> |
conftool action : set/pooled=yes; selector: name=chromium.wikimedia.org |
[production] |
|
2017-11-21
§
|
| 23:55 |
<bblack@neodymium> |
conftool action : set/pooled=no; selector: name=chromium.wikimedia.org |
[production] |
| 23:30 |
<mholloway-shell@tin> |
Finished deploy [mobileapps/deploy@fc01242]: Update mobileapps to 52d6a83 (duration: 04m 36s) |
[production] |
| 23:26 |
<mholloway-shell@tin> |
Started deploy [mobileapps/deploy@fc01242]: Update mobileapps to 52d6a83 |
[production] |
| 22:45 |
<bblack@neodymium> |
conftool action : set/pooled=yes; selector: name=achernar.wikimedia.org |
[production] |
| 22:38 |
<bblack@neodymium> |
conftool action : set/pooled=no; selector: name=achernar.wikimedia.org |
[production] |
| 22:36 |
<bblack@neodymium> |
conftool action : set/pooled=yes; selector: name=hydrogen.wikimedia.org |
[production] |
| 22:29 |
<urandom> |
Bootstrapping Cassandra, restbase2004-b.codfw.wmnet (T179422) |
[production] |
| 22:12 |
<mutante> |
gerrit - temp disable puppet on cobalt (prod gerrit), test switching gerrit logging to logstash on gerrit2001 - gerrit:392079 gerrit:392083 T141324 |
[production] |
| 22:05 |
<bblack@neodymium> |
conftool action : set/pooled=no; selector: name=hydrogen.wikimedia.org |
[production] |
| 22:03 |
<bblack@neodymium> |
conftool action : set/pooled=yes; selector: name=acamar.wikimedia.org |
[production] |
| 22:02 |
<bblack> |
repooling acamar for recdns |
[production] |
| 21:26 |
<ariel@tin> |
Finished deploy [dumps/dumps@16f92d6]: take 2: gzip namespace and abstract dumps; remove last configfile existence checks (duration: 00m 02s) |
[production] |
| 21:26 |
<ariel@tin> |
Started deploy [dumps/dumps@16f92d6]: take 2: gzip namespace and abstract dumps; remove last configfile existence checks |
[production] |
| 20:45 |
<bblack> |
recdns: puppet disabled on all, acamar depooled, careful deploys going on for anycast+recdns stuff |
[production] |
| 20:42 |
<ayounsi@neodymium> |
conftool action : set/pooled=no; selector: name=acamar.wikimedia.org |
[production] |
| 20:26 |
<ariel@tin> |
Finished deploy [dumps/dumps@16f92d6]: gzip namespace and abstract dumps; remove last configfile existence checks (duration: 00m 16s) |
[production] |