|
2017-06-13
ยง
|
| 21:59 |
<gwicke> |
restarted pdfrender on scb1003; was spinning on CPU & using 15G of memory (!) |
[production] |
| 21:58 |
<gwicke> |
restarted pdfrender on scb1002 and scb1004; was spinning on CPU |
[production] |
| 21:56 |
<hashar> |
Zuul back, running in an interactive terminal. |
[production] |
| 21:46 |
<mutante> |
netmon1002 - was able to "keyholder arm" after stretch install after applying https://gerrit.wikimedia.org/r/358884 as hotfix |
[production] |
| 21:30 |
<mobrovac@tin> |
Finished deploy [restbase/deploy@9a86d4c]: (no justification provided) (duration: 01m 06s) |
[production] |
| 21:29 |
<mobrovac@tin> |
Started deploy [restbase/deploy@9a86d4c]: (no justification provided) |
[production] |
| 21:13 |
<hashar> |
Gracefully restarting Zuul |
[production] |
| 21:13 |
<hashar> |
Gracefully restarting Zuul |
[releng] |
| 21:11 |
<ppchelko@tin> |
Finished deploy [changeprop/deploy@4ba3c59]: Rate-limiter enhancements (duration: 01m 08s) |
[production] |
| 21:10 |
<ppchelko@tin> |
Started deploy [changeprop/deploy@4ba3c59]: Rate-limiter enhancements |
[production] |
| 21:02 |
<demon@tin> |
Synchronized php-1.30.0-wmf.5/extensions/CentralAuth/includes/CentralAuthHooks.php: Fix bad method name (duration: 00m 44s) |
[production] |
| 20:37 |
<hashar> |
Restarting Nodepool. apparently confused in pool tracking and spawning to many Trusty nodes (7 instead of 4) |
[releng] |
| 20:37 |
<hashar> |
Restarting Nodepool. apparently confused in pool tracking and spawning to many Trusty nodes (7 instead of 4) |
[production] |
| 20:31 |
<hashar> |
Nodepool: deleted a bunch of Trusty instances. It scheduled lot of them that are taking slots in the pool. Better have jessie nodes to be spawned instead since there is high demand for them |
[releng] |
| 20:19 |
<hashar> |
deployment-prep: added Polishdeveloper to the "importer" global group. https://deployment.wikimedia.beta.wmflabs.org/wiki/Special:GlobalUserRights/Polishdeveloper - T167823 |
[releng] |
| 20:02 |
<demon@tin> |
Synchronized php-1.30.0-wmf.5/includes/api/ApiParse.php: T167826 (duration: 00m 44s) |
[production] |
| 20:00 |
<mobrovac@tin> |
Finished deploy [restbase/deploy@4c1cdd0]: (no justification provided) (duration: 04m 29s) |
[production] |
| 19:56 |
<mobrovac@tin> |
Started deploy [restbase/deploy@4c1cdd0]: (no justification provided) |
[production] |
| 19:37 |
<Amir1> |
restarting ores-related services in scb1001 (T167819) |
[production] |
| 19:24 |
<mutante> |
scb1001 - killed process 10971 (pdfrendering/electron) |
[production] |
| 19:24 |
<demon@tin> |
Synchronized php-1.30.0-wmf.5/extensions/CategoryTree/CategoryPageSubclass.php: Fix up variable visibility (duration: 00m 44s) |
[production] |
| 19:12 |
<demon@tin> |
rebuilt wikiversions.php and synchronized wikiversions files: group0 to wmf.5 |
[production] |
| 19:09 |
<mobrovac@tin> |
Finished deploy [restbase/deploy@9a86d4c]: (no justification provided) (duration: 07m 33s) |
[production] |
| 19:08 |
<mutante> |
netmon1002 - reinstallled with stretch, revoked puppet cert, salt key, signing new cert, accepting new key, initial puppet run (T159756) |
[production] |
| 19:01 |
<mobrovac@tin> |
Started deploy [restbase/deploy@9a86d4c]: (no justification provided) |
[production] |
| 18:56 |
<mutante> |
reinstalling netmon1002 with stretch - scheduled icinga downtime |
[production] |
| 18:54 |
<legoktm> |
starting to delete all rows from linter tables on large wikis - T167758 |
[production] |
| 18:48 |
<mobrovac@tin> |
Finished deploy [restbase/deploy@4c1cdd0]: (no justification provided) (duration: 04m 36s) |
[production] |
| 18:47 |
<andrewbogott> |
root@deployment-salt02:~# salt "*" cmd.run "apt-get -y install facter" |
[deployment-prep] |
| 18:47 |
<andrewbogott> |
root@deployment-salt02:~# salt "*" cmd.run "apt-get -y install facter" |
[releng] |
| 18:46 |
<andrewbogott> |
using salt to "apt-get -y install facter" on all deployment-prep instances |
[releng] |
| 18:43 |
<mobrovac@tin> |
Started deploy [restbase/deploy@4c1cdd0]: (no justification provided) |
[production] |
| 18:39 |
<mobrovac@tin> |
Started deploy [restbase/deploy@4c1cdd0]: (no justification provided) |
[production] |
| 18:38 |
<andrewbogott> |
restarting apache2 on deployment-puppetmaster02 |
[releng] |
| 18:37 |
<mobrovac@tin> |
Finished deploy [restbase/deploy@4c1cdd0]: (no justification provided) (duration: 04m 19s) |
[production] |
| 18:37 |
<andrewbogott> |
doing a git fetch and rebase for deployment-puppetmaster02 |
[releng] |
| 18:33 |
<mobrovac@tin> |
Started deploy [restbase/deploy@4c1cdd0]: (no justification provided) |
[production] |
| 18:27 |
<demon@tin> |
Finished scap: testwiki to wmf.5 + l10n bootstrap (duration: 42m 16s) |
[production] |
| 17:52 |
<bblack> |
cp4021 reboot for bnx2x modparam change |
[production] |
| 17:50 |
<ottomata> |
merged removal of x_forwarded_for from all varnishkafka webrequest instances |
[production] |
| 17:45 |
<ladsgroup@tin> |
Finished deploy [ores/deploy@862aea9]: ORES deploy early June: T167223 (duration: 33m 52s) |
[production] |
| 17:45 |
<demon@tin> |
Started scap: testwiki to wmf.5 + l10n bootstrap |
[production] |
| 17:42 |
<demon@tin> |
Pruned MediaWiki: 1.30.0-wmf.2 [keeping static files] (duration: 01m 13s) |
[production] |
| 17:40 |
<demon@tin> |
Pruned MediaWiki: 1.30.0-wmf.1 [keeping static files] (duration: 05m 10s) |
[production] |
| 17:39 |
<bblack> |
restart varnish-be on cp2002 (mailbox lag, likely induced by swift traffic testing in codfw) |
[production] |
| 17:11 |
<ladsgroup@tin> |
Started deploy [ores/deploy@862aea9]: ORES deploy early June: T167223 |
[production] |
| 17:06 |
<akosiaris> |
rebooting sca2003 for tests |
[production] |
| 17:00 |
<elukey> |
hacking apache on mediawiki05 to test rewrite rules |
[releng] |
| 16:35 |
<moritzm> |
upgrading osmium to HHVM 3.18 |
[production] |
| 16:34 |
<Amir1> |
deploying a24433f to prod |
[wikilabels] |