1751-1800 of 10000 results (43ms)
2017-06-14 §
01:48 <mutante> netmon1002 - chown rancid:rancid /var/lib/rancid ; touch /var/lib/rancid/.gitconfig, let rancid write to config, then git config --global user.email and user.name as the rancid user | fix permissions on .git/objects files, let rancid user own them all | re-commit .gitingore change | SSH_AUTH_SOCK=/run/keyholder/proxy.sock /usr/lib/rancid/bin/rancid-run as user "rancid" runs clean, [production]
01:20 <mutante> netmon1002 - copied missing router.db, routers.all/.down/.up over from netmon1001 to /var/lib/rancid/core. routers.db is an untracked file, the others are in .gitignore. this is all like on netmon1001 as well. adding routers.db to .gitignore file on both, like the other router* files already were (T159756) [production]
01:00 <mutante> netmon1002 - locally "git clone /var/lib/rancid/GIT/core" into /var/lib/rancid (i rsynced that but it's a bare repository without a work tree. work tree is /var/lib/rancid/core (after this) (T159756) [production]
00:44 <mutante> naos: disarm keyholder and armed it again to proof i didn't break anything on jessie by fixing keyholder on stretch with gerrit:358884 [production]
00:39 <demon@tin> Synchronized wmf-config/CommonSettings.php: extdist update (duration: 00m 44s) [production]
00:09 <aaron@tin> Synchronized wmf-config/InitialiseSettings.php: Capture messages on 'autoloader' debug log channel (duration: 00m 44s) [production]
2017-06-13 §
23:29 <RainbowSprinkles> gerrit: upgrading on master 2.13.4-13-gc0c5cc4742 -> 2.13.8-1-g7c438d37a2 (been running on slave for a week) [production]
23:13 <mutante> contint1001 - started zuul using the old init script [production]
23:04 <mutante> netmon1001/1002: rsynced /var/lib/rancid/CVS and /var/lib/rancid/GIT from 1001 to 1002 for rancid migration (T159756) [production]
23:04 <demon@tin> Synchronized php-1.30.0-wmf.4/extensions/OpenStackManager: Re-adding deleted special page (duration: 00m 45s) [production]
22:06 <ejegg> updated fundraising tools from f2522cdabf1741a60b7b60ac8f7ead7afd50b054 to 585f546aa0c092ccd938a9a01b4bc3eb7662804d [production]
21:59 <gwicke> restarted pdfrender on scb1003; was spinning on CPU & using 15G of memory (!) [production]
21:58 <gwicke> restarted pdfrender on scb1002 and scb1004; was spinning on CPU [production]
21:56 <hashar> Zuul back, running in an interactive terminal. [production]
21:46 <mutante> netmon1002 - was able to "keyholder arm" after stretch install after applying https://gerrit.wikimedia.org/r/358884 as hotfix [production]
21:30 <mobrovac@tin> Finished deploy [restbase/deploy@9a86d4c]: (no justification provided) (duration: 01m 06s) [production]
21:29 <mobrovac@tin> Started deploy [restbase/deploy@9a86d4c]: (no justification provided) [production]
21:13 <hashar> Gracefully restarting Zuul [production]
21:11 <ppchelko@tin> Finished deploy [changeprop/deploy@4ba3c59]: Rate-limiter enhancements (duration: 01m 08s) [production]
21:10 <ppchelko@tin> Started deploy [changeprop/deploy@4ba3c59]: Rate-limiter enhancements [production]
21:02 <demon@tin> Synchronized php-1.30.0-wmf.5/extensions/CentralAuth/includes/CentralAuthHooks.php: Fix bad method name (duration: 00m 44s) [production]
20:37 <hashar> Restarting Nodepool. apparently confused in pool tracking and spawning to many Trusty nodes (7 instead of 4) [production]
20:02 <demon@tin> Synchronized php-1.30.0-wmf.5/includes/api/ApiParse.php: T167826 (duration: 00m 44s) [production]
20:00 <mobrovac@tin> Finished deploy [restbase/deploy@4c1cdd0]: (no justification provided) (duration: 04m 29s) [production]
19:56 <mobrovac@tin> Started deploy [restbase/deploy@4c1cdd0]: (no justification provided) [production]
19:37 <Amir1> restarting ores-related services in scb1001 (T167819) [production]
19:24 <mutante> scb1001 - killed process 10971 (pdfrendering/electron) [production]
19:24 <demon@tin> Synchronized php-1.30.0-wmf.5/extensions/CategoryTree/CategoryPageSubclass.php: Fix up variable visibility (duration: 00m 44s) [production]
19:12 <demon@tin> rebuilt wikiversions.php and synchronized wikiversions files: group0 to wmf.5 [production]
19:09 <mobrovac@tin> Finished deploy [restbase/deploy@9a86d4c]: (no justification provided) (duration: 07m 33s) [production]
19:08 <mutante> netmon1002 - reinstallled with stretch, revoked puppet cert, salt key, signing new cert, accepting new key, initial puppet run (T159756) [production]
19:01 <mobrovac@tin> Started deploy [restbase/deploy@9a86d4c]: (no justification provided) [production]
18:56 <mutante> reinstalling netmon1002 with stretch - scheduled icinga downtime [production]
18:54 <legoktm> starting to delete all rows from linter tables on large wikis - T167758 [production]
18:48 <mobrovac@tin> Finished deploy [restbase/deploy@4c1cdd0]: (no justification provided) (duration: 04m 36s) [production]
18:43 <mobrovac@tin> Started deploy [restbase/deploy@4c1cdd0]: (no justification provided) [production]
18:39 <mobrovac@tin> Started deploy [restbase/deploy@4c1cdd0]: (no justification provided) [production]
18:37 <mobrovac@tin> Finished deploy [restbase/deploy@4c1cdd0]: (no justification provided) (duration: 04m 19s) [production]
18:33 <mobrovac@tin> Started deploy [restbase/deploy@4c1cdd0]: (no justification provided) [production]
18:27 <demon@tin> Finished scap: testwiki to wmf.5 + l10n bootstrap (duration: 42m 16s) [production]
17:52 <bblack> cp4021 reboot for bnx2x modparam change [production]
17:50 <ottomata> merged removal of x_forwarded_for from all varnishkafka webrequest instances [production]
17:45 <ladsgroup@tin> Finished deploy [ores/deploy@862aea9]: ORES deploy early June: T167223 (duration: 33m 52s) [production]
17:45 <demon@tin> Started scap: testwiki to wmf.5 + l10n bootstrap [production]
17:42 <demon@tin> Pruned MediaWiki: 1.30.0-wmf.2 [keeping static files] (duration: 01m 13s) [production]
17:40 <demon@tin> Pruned MediaWiki: 1.30.0-wmf.1 [keeping static files] (duration: 05m 10s) [production]
17:39 <bblack> restart varnish-be on cp2002 (mailbox lag, likely induced by swift traffic testing in codfw) [production]
17:11 <ladsgroup@tin> Started deploy [ores/deploy@862aea9]: ORES deploy early June: T167223 [production]
17:06 <akosiaris> rebooting sca2003 for tests [production]
16:35 <moritzm> upgrading osmium to HHVM 3.18 [production]