|
2017-06-13
ยง
|
| 23:29 |
<RainbowSprinkles> |
gerrit: upgrading on master 2.13.4-13-gc0c5cc4742 -> 2.13.8-1-g7c438d37a2 (been running on slave for a week) |
[production] |
| 23:13 |
<mutante> |
contint1001 - started zuul using the old init script |
[production] |
| 23:04 |
<mutante> |
netmon1001/1002: rsynced /var/lib/rancid/CVS and /var/lib/rancid/GIT from 1001 to 1002 for rancid migration (T159756) |
[production] |
| 23:04 |
<demon@tin> |
Synchronized php-1.30.0-wmf.4/extensions/OpenStackManager: Re-adding deleted special page (duration: 00m 45s) |
[production] |
| 22:06 |
<ejegg> |
updated fundraising tools from f2522cdabf1741a60b7b60ac8f7ead7afd50b054 to 585f546aa0c092ccd938a9a01b4bc3eb7662804d |
[production] |
| 22:05 |
<hashar> |
Zuul resarted manually from a terminal on contint1001. It does not have any statsd configuration so we will miss metrics for a bit till it is restarted properly. |
[releng] |
| 21:59 |
<gwicke> |
restarted pdfrender on scb1003; was spinning on CPU & using 15G of memory (!) |
[production] |
| 21:58 |
<gwicke> |
restarted pdfrender on scb1002 and scb1004; was spinning on CPU |
[production] |
| 21:56 |
<hashar> |
Zuul back, running in an interactive terminal. |
[production] |
| 21:46 |
<mutante> |
netmon1002 - was able to "keyholder arm" after stretch install after applying https://gerrit.wikimedia.org/r/358884 as hotfix |
[production] |
| 21:30 |
<mobrovac@tin> |
Finished deploy [restbase/deploy@9a86d4c]: (no justification provided) (duration: 01m 06s) |
[production] |
| 21:29 |
<mobrovac@tin> |
Started deploy [restbase/deploy@9a86d4c]: (no justification provided) |
[production] |
| 21:13 |
<hashar> |
Gracefully restarting Zuul |
[production] |
| 21:13 |
<hashar> |
Gracefully restarting Zuul |
[releng] |
| 21:11 |
<ppchelko@tin> |
Finished deploy [changeprop/deploy@4ba3c59]: Rate-limiter enhancements (duration: 01m 08s) |
[production] |
| 21:10 |
<ppchelko@tin> |
Started deploy [changeprop/deploy@4ba3c59]: Rate-limiter enhancements |
[production] |
| 21:02 |
<demon@tin> |
Synchronized php-1.30.0-wmf.5/extensions/CentralAuth/includes/CentralAuthHooks.php: Fix bad method name (duration: 00m 44s) |
[production] |
| 20:37 |
<hashar> |
Restarting Nodepool. apparently confused in pool tracking and spawning to many Trusty nodes (7 instead of 4) |
[releng] |
| 20:37 |
<hashar> |
Restarting Nodepool. apparently confused in pool tracking and spawning to many Trusty nodes (7 instead of 4) |
[production] |
| 20:31 |
<hashar> |
Nodepool: deleted a bunch of Trusty instances. It scheduled lot of them that are taking slots in the pool. Better have jessie nodes to be spawned instead since there is high demand for them |
[releng] |
| 20:19 |
<hashar> |
deployment-prep: added Polishdeveloper to the "importer" global group. https://deployment.wikimedia.beta.wmflabs.org/wiki/Special:GlobalUserRights/Polishdeveloper - T167823 |
[releng] |
| 20:02 |
<demon@tin> |
Synchronized php-1.30.0-wmf.5/includes/api/ApiParse.php: T167826 (duration: 00m 44s) |
[production] |
| 20:00 |
<mobrovac@tin> |
Finished deploy [restbase/deploy@4c1cdd0]: (no justification provided) (duration: 04m 29s) |
[production] |
| 19:56 |
<mobrovac@tin> |
Started deploy [restbase/deploy@4c1cdd0]: (no justification provided) |
[production] |
| 19:37 |
<Amir1> |
restarting ores-related services in scb1001 (T167819) |
[production] |
| 19:24 |
<mutante> |
scb1001 - killed process 10971 (pdfrendering/electron) |
[production] |
| 19:24 |
<demon@tin> |
Synchronized php-1.30.0-wmf.5/extensions/CategoryTree/CategoryPageSubclass.php: Fix up variable visibility (duration: 00m 44s) |
[production] |
| 19:12 |
<demon@tin> |
rebuilt wikiversions.php and synchronized wikiversions files: group0 to wmf.5 |
[production] |
| 19:09 |
<mobrovac@tin> |
Finished deploy [restbase/deploy@9a86d4c]: (no justification provided) (duration: 07m 33s) |
[production] |
| 19:08 |
<mutante> |
netmon1002 - reinstallled with stretch, revoked puppet cert, salt key, signing new cert, accepting new key, initial puppet run (T159756) |
[production] |
| 19:01 |
<mobrovac@tin> |
Started deploy [restbase/deploy@9a86d4c]: (no justification provided) |
[production] |
| 18:56 |
<mutante> |
reinstalling netmon1002 with stretch - scheduled icinga downtime |
[production] |
| 18:54 |
<legoktm> |
starting to delete all rows from linter tables on large wikis - T167758 |
[production] |
| 18:48 |
<mobrovac@tin> |
Finished deploy [restbase/deploy@4c1cdd0]: (no justification provided) (duration: 04m 36s) |
[production] |
| 18:47 |
<andrewbogott> |
root@deployment-salt02:~# salt "*" cmd.run "apt-get -y install facter" |
[deployment-prep] |
| 18:47 |
<andrewbogott> |
root@deployment-salt02:~# salt "*" cmd.run "apt-get -y install facter" |
[releng] |
| 18:46 |
<andrewbogott> |
using salt to "apt-get -y install facter" on all deployment-prep instances |
[releng] |
| 18:43 |
<mobrovac@tin> |
Started deploy [restbase/deploy@4c1cdd0]: (no justification provided) |
[production] |
| 18:39 |
<mobrovac@tin> |
Started deploy [restbase/deploy@4c1cdd0]: (no justification provided) |
[production] |
| 18:38 |
<andrewbogott> |
restarting apache2 on deployment-puppetmaster02 |
[releng] |
| 18:37 |
<mobrovac@tin> |
Finished deploy [restbase/deploy@4c1cdd0]: (no justification provided) (duration: 04m 19s) |
[production] |
| 18:37 |
<andrewbogott> |
doing a git fetch and rebase for deployment-puppetmaster02 |
[releng] |
| 18:33 |
<mobrovac@tin> |
Started deploy [restbase/deploy@4c1cdd0]: (no justification provided) |
[production] |
| 18:27 |
<demon@tin> |
Finished scap: testwiki to wmf.5 + l10n bootstrap (duration: 42m 16s) |
[production] |
| 17:52 |
<bblack> |
cp4021 reboot for bnx2x modparam change |
[production] |
| 17:50 |
<ottomata> |
merged removal of x_forwarded_for from all varnishkafka webrequest instances |
[production] |
| 17:45 |
<ladsgroup@tin> |
Finished deploy [ores/deploy@862aea9]: ORES deploy early June: T167223 (duration: 33m 52s) |
[production] |
| 17:45 |
<demon@tin> |
Started scap: testwiki to wmf.5 + l10n bootstrap |
[production] |
| 17:42 |
<demon@tin> |
Pruned MediaWiki: 1.30.0-wmf.2 [keeping static files] (duration: 01m 13s) |
[production] |
| 17:40 |
<demon@tin> |
Pruned MediaWiki: 1.30.0-wmf.1 [keeping static files] (duration: 05m 10s) |
[production] |