1-50 of 79 results (11ms)
2017-01-04 §
21:29 <mutante> deployment-cache-text-04 - running acme-setup command to debug .. Creating CSR /etc/acme/csr/beta_wmflabs_org.pem [deployment-prep]
21:26 <Krenair> trying to troubleshoot puppet by stopping nginx then letting puppet start it [deployment-prep]
21:05 <mutante> deployment-cache-text04 stopping nginx service, running puppet to debug dependency issue [deployment-prep]
2016-12-19 §
21:21 <andrewbogott> and also python-functools32_3.2.3.2-3~bpo8+1_all.deb [deployment-prep]
21:20 <andrewbogott> upgrading to python-jsonschema_2.5.1-5~bpo8+1_all.deb on deployment-eventlogging03 [deployment-prep]
20:51 <andrewbogott> upgrading to python-requests_2.12.3-1_all.deb ./python-urllib3_1.19.1-1_all.deb on deployment-mediawiki04 and deployment-tin [deployment-prep]
2016-12-04 §
15:25 <Krenair> Found a git-sync-upstream cron on deployment-mx for some reason... commented for now, but wtf was this doing on a MX server? [deployment-prep]
2016-11-23 §
15:04 <Krenair> fixed puppet on deployment-cache-text04 by manually enabling experimental apt repo, see T150660 [deployment-prep]
2016-11-16 §
20:02 <Krenair> mysql master back up, root identity is now unix socket based rather than password [deployment-prep]
19:57 <Krenair> taking mysql master down to fix perms [deployment-prep]
07:52 <Krenair> the new mysql root password for -db04 is at /tmp/newmysqlpass as well as in a new file in the puppetmaster's labs/private.git [deployment-prep]
2016-11-09 §
20:27 <Krenair> removed default SSH access from production host 208.80.154.135, the old gallium IP [deployment-prep]
2016-11-03 §
05:04 <Krenair> beginning to move the rest of beta to the new puppetmaster [deployment-prep]
2016-11-02 §
18:51 <Krenair> armed keyholder on -tin and -mira [deployment-prep]
18:50 <Krenair> started mysql on -db boxes to bring beta back online [deployment-prep]
2016-11-01 §
22:22 <Krenair> started mysql on -db03 to hopefully pull us out of read-only mode [deployment-prep]
22:21 <Krenair> started mysql on -db04 [deployment-prep]
22:19 <Krenair> stopped and started udp2log-mw on -fluorine02 [deployment-prep]
22:00 <Krenair> started moving nodes back to the new puppetmaster [deployment-prep]
02:55 <Krenair> Managed to mess up the deployment-puppetmaster02 cert, had to move those nodes back [deployment-prep]
2016-10-31 §
20:57 <Krenair> moving some nodes to deployment-puppetmaster02 [deployment-prep]
16:57 <bd808> Added Niharika29 as project member [deployment-prep]
2016-10-27 §
18:46 <bd808> Testing dual page wiki logging by stashbot. (check #3) [deployment-prep]
18:36 <bd808> Testing dual page wiki logging by stashbot. (second attempt) [deployment-prep]
18:14 <bd808> Testing dual page wiki logging by stashbot. [deployment-prep]
2016-08-02 §
14:02 <gehel> rebooting deployment-elastic06 (unresponsive to SSH and Salt) [deployment-prep]
02:51 <Krenair> https://deployment.wikimedia.beta.wmflabs, https://meta.wikimedia.beta.wmflabs, and their mobile variants now also have valid certs and TLS redirects. [deployment-prep]
01:11 <Krenair> Proper SSL certificate up at https://upload.beta.wmflabs.org - HTTP has been changed to force TLS redirect. [deployment-prep]
2016-08-01 §
20:58 <Krenair> deleted 2014/2015 files from deployment-stream:/var/log/diamond to get space on /var and stop it warning [deployment-prep]
2016-07-27 §
06:07 <TimStarling> fixed broken puppet git checkout on deployment-puppetmaster, updated [deployment-prep]
2016-07-13 §
20:45 <Krenair> RIP NFS [deployment-prep]
2016-07-11 §
23:24 <Krenair> Unmounted /data/project (NFS) on all active hosts (mediawiki0[1-3], jobrunner01, tmh01), leaving just deployment-upload (shutoff, to schedule for deletion soon) - T64835 [deployment-prep]
2016-07-09 §
00:46 <Krenair> T64835: `mwscript extensions/WikimediaMaintenance/filebackend/setZoneAccess.php zerowiki --backend=local-multiwrite --private` [deployment-prep]
00:46 <Krenair> T64835: `foreachwikiindblist "% all-labs.dblist - private.dblist" extensions/WikimediaMaintenance/filebackend/setZoneAccess.php --backend=local-multiwrite` [deployment-prep]
00:46 <Krenair> T64835: Live-hacked some temporary swift config in [deployment-prep]
2016-06-27 §
22:32 <eberhardson> deployed gerrit.wikimedia.org/r/296279 to puppetmaster to test kibana4 role [deployment-prep]
2016-06-25 §
03:24 <Krenair> Changed eventbus key in secrets (from being a symlink to eventlogging to being a new random key) so check_keyholder works again [deployment-prep]
2016-06-22 §
22:23 <Krenair> Installed netpbm on all deployment-mediawiki* hosts to fix ProofreadPage thumbnailing. I wonder if we should include the puppet mediawiki::packages::multimedia class on these hosts really [deployment-prep]
2016-06-13 §
16:06 <Krenair> Rebooted deployment-ircd, it was stuck somehow [deployment-prep]
13:53 <yuvipanda> kicked deployment-salt via nova for Krenair [deployment-prep]
13:35 <Krenair> Fixed puppet on -tin by symlinking eventbus key to eventlogging in -puppetmaster:/var/lib/git/labs/private/modules/secret/secrets/keyholder [deployment-prep]
2016-06-01 §
02:14 <Krenair> Started redis-server on deployment-rcstream to stop MW hhvm.log spam [deployment-prep]
2016-05-09 §
15:39 <andrewbogott> migrating deployment-flourine to labvirt1009 [deployment-prep]
2016-05-03 §
01:41 <Krenair> ran package updates on deployment-parsoid06 so that exim4 would start so puppet will run [deployment-prep]
2016-05-02 §
09:54 <gehel> restart elasticsearch cluster to ensure multicast configuration is disabled (T110236) [deployment-prep]
2016-04-10 §
06:04 <Krenair> deleted some large files under deployment-mediawiki01:/var/log/nutcracker to free up space on / [deployment-prep]
2016-04-09 §
16:08 <Krenair> (same for -conf03, -sentry01, -redis01, -upload - some of these are now fully fixed and some are better than they were before) [deployment-prep]
15:59 <Krenair> mostly fixed puppet on deployment-sca02 by changing /etc/puppet/puppet.conf to use project name as part of puppetmaster's hostname [deployment-prep]
15:56 <Krenair> fixed broken /etc/puppet/puppet.conf on deployment-cache-text04 (it started with a copy of the file for the labs central puppetmaster and then had the correct version pointing to the project's puppetmaster) [deployment-prep]
15:47 <Krenair> reenabled puppet on eventlogging04 as no reason was provided for disabling, first run successful [deployment-prep]