|
2017-09-20
ยง
|
| 21:09 |
<bblack@neodymium> |
conftool action : set/pooled=no; selector: name=cp4015.ulsfo.wmnet |
[production] |
| 20:58 |
<bblack@neodymium> |
conftool action : set/pooled=no; selector: name=cp4007.ulsfo.wmnet |
[production] |
| 20:57 |
<paladox> |
disabling puppet on gerrit-test3 temp, trying to reproduce the slave issue in prod for gerrit |
[git] |
| 20:40 |
<bsitzmann@tin> |
Finished deploy [mobileapps/deploy@e23bf66]: Update mobileapps to d46860e (T175759 T176263) (duration: 06m 05s) |
[production] |
| 20:36 |
<bblack@neodymium> |
conftool action : set/pooled=no; selector: name=cp4014.ulsfo.wmnet |
[production] |
| 20:35 |
<ebernhar|lunch> |
increase max recovery bytes/s to 80mb in eqiad elasitcsearch cluster |
[production] |
| 20:34 |
<bsitzmann@tin> |
Started deploy [mobileapps/deploy@e23bf66]: Update mobileapps to d46860e (T175759 T176263) |
[production] |
| 20:33 |
<demon@tin> |
Synchronized php-1.30.0-wmf.19/includes/diff/DifferenceEngine.php: fix some warnings (duration: 00m 51s) |
[production] |
| 19:55 |
<bblack@neodymium> |
conftool action : set/pooled=no; selector: name=cp4016.ulsfo.wmnet |
[production] |
| 19:54 |
<eevans@tin> |
Finished deploy [cassandra/metrics-collector@df909a1] (dev-cluster): Deploy 4.1.0 artifact to dev cluster (duration: 00m 10s) |
[production] |
| 19:54 |
<eevans@tin> |
Started deploy [cassandra/metrics-collector@df909a1] (dev-cluster): Deploy 4.1.0 artifact to dev cluster |
[production] |
| 19:53 |
<bblack@neodymium> |
conftool action : set/pooled=no; selector: name=cp4006.ulsfo.wmnet |
[production] |
| 19:48 |
<eevans@tin> |
Finished deploy [cassandra/metrics-collector@df909a1]: Pushing out 4.1.0 release (duration: 00m 47s) |
[production] |
| 19:47 |
<eevans@tin> |
Started deploy [cassandra/metrics-collector@df909a1]: Pushing out 4.1.0 release |
[production] |
| 19:39 |
<bblack@neodymium> |
conftool action : set/pooled=no; selector: name=cp4008.ulsfo.wmnet |
[production] |
| 19:26 |
<demon@tin> |
rebuilt wikiversions.php and synchronized wikiversions files: group1 to wmf.19 |
[production] |
| 19:24 |
<demon@tin> |
Synchronized php: symlink swap for wmf.19 (duration: 00m 49s) |
[production] |
| 19:02 |
<catrope@tin> |
Finished scap: Patches for T176302, T175962, T173533 (duration: 21m 29s) |
[production] |
| 18:40 |
<catrope@tin> |
Started scap: Patches for T176302, T175962, T173533 |
[production] |
| 17:29 |
<hoo> |
Started mwscript refreshLinks.php --wiki elwiki --namespace 0 in a screen on terbium. Can be KILLed at any time, if needed. |
[production] |
| 17:29 |
<hoo> |
Started mwscript refreshLinks.php --wiki elwiki --namespace 0 in a screen on elwiki. Can be KILLed at any time, if needed. |
[production] |
| 17:13 |
<hoo> |
Ran mwscript refreshLinks.php --wiki elwiki --namespace 0 -e 30 for testing purposes |
[production] |
| 17:06 |
<ejegg> |
removed limit from donation queue consumer |
[production] |
| 17:05 |
<otto@tin> |
Finished deploy [eventstreams/deploy@e62ab64]: (no justification provided) (duration: 04m 44s) |
[production] |
| 17:01 |
<otto@tin> |
Started deploy [eventstreams/deploy@e62ab64]: (no justification provided) |
[production] |
| 16:53 |
<madhuvishy> |
Upgraded and restarted apache2 on labs-puppetmaster.wikimedia.org |
[production] |
| 16:52 |
<madhuvishy> |
apt-get install --only-upgrade apache2; service apache2 restart on tools-puppetmaster-01 |
[tools] |
| 16:45 |
<gehel@puppetmaster1001> |
conftool action : set/pooled=yes; selector: dc=eqiad,cluster=elasticsearch |
[production] |
| 16:42 |
<gehel> |
rolling restart of logstash ingesters after cirrus eqiad recovery |
[production] |
| 16:34 |
<gehel> |
elasticsearch eqiad is finally recovering, after the restart of elastic1051 |
[production] |
| 16:34 |
<gehel> |
cleanup jar hell and restart elastic1051 |
[production] |
| 16:19 |
<gehel> |
starting elasticsearch on all nodes on eqiad |
[production] |
| 16:10 |
<gehel> |
restarting elasticsearch masters on eqiad (elastic1030, 36 and 40) |
[production] |
| 16:04 |
<gehel@puppetmaster1001> |
conftool action : set/pooled=no; selector: dc=eqiad,cluster=elasticsearch |
[production] |
| 16:03 |
<gehel> |
depooling all nodes in elasticsearch eqiad to investigate failed cluster restart (traffic is already directed to codfw) |
[production] |
| 15:46 |
<addshore> |
reloading zuul for https://gerrit.wikimedia.org/r/#/c/379250/ |
[releng] |
| 15:46 |
<ejegg> |
limited donations queue consumer to 100 per run |
[production] |
| 15:36 |
<ejegg> |
updated payments-wiki from 896f16ac380aaa6022749b395f9187ecf4bc802b to 9da84823ff4ac3071d218ec211fb59c054de9e15 |
[production] |
| 15:36 |
<gehel> |
upgrading elasticsearch plugins on elasticsearch eqiad, including cold restart of the cluster - T173231 |
[production] |
| 15:33 |
<ejegg> |
limited donations queue consumer to 200 per run |
[production] |
| 14:03 |
<akosiaris@tin> |
Synchronized wmf-config/InitialiseSettings.php: Whitelist wtp10[25-48] for Linter (duration: 00m 49s) |
[production] |
| 13:59 |
<addshore> |
docker push docker.io/wmfreleng/mediawiki-phan:v2017.09.20.13.49 & latest # built from master |
[releng] |
| 13:59 |
<addshore> |
docker push docker.io/wmfreleng/composer:v2017.09.20.13.44 & latest # built from master |
[releng] |
| 13:59 |
<addshore> |
docker push docker.io/wmfreleng/zuul-cloner:v2017.09.20.13.44 & latest # built from master |
[releng] |
| 13:59 |
<addshore> |
docker push docker.io/wmfreleng/php-mediawiki:v2017.09.20.13.43 & latest # built from master |
[releng] |
| 13:59 |
<addshore> |
docker push docker.io/wmfreleng/php:v2017.09.20.13.40 & latest # built from master |
[releng] |
| 13:42 |
<otto@tin> |
Synchronized /srv/mediawiki-staging/wmf-config/CommonSettings-labs.php: (no justification provided) (duration: 00m 49s) |
[production] |
| 13:24 |
<ema> |
begin rolling codfw varnish-be restarts, 20m spacing |
[production] |
| 13:20 |
<dereckson@tin> |
Synchronized wmf-config/CommonSettings.php: Switch elasticsearch active cluster to codfw ([[Gerrit:378850]], 3/3) (duration: 00m 48s) |
[production] |
| 13:18 |
<dereckson@tin> |
Synchronized wmf-config/InitialiseSettings.php: Switch elasticsearch active cluster to codfw ([[Gerrit:378850]], 2/3) (duration: 00m 49s) |
[production] |