3751-3800 of 10000 results (33ms)
2016-02-16 §
11:00 <joal> Deploying Refinery from tin (trying) [analytics]
10:50 <godog> start swiftrepl commons thumbs for top50 popular size T125791 [production]
10:14 <akosiaris> disable puppet, stop salt-minion on sca100{1,2} [production]
09:52 <hashar> will cut the wmf branches this afternoon starting around 14:00 CET [releng]
09:52 <hashar> will cut the wmf branches this afternoon starting around 14:00 CET [production]
09:31 <hoo@tin> Synchronized wmf-config/InitialiseSettings-labs.php: (no message) (duration: 00m 58s) [production]
02:34 <l10nupdate@tin> ResourceLoader cache refresh completed at Tue Feb 16 02:34:30 UTC 2016 (duration 8m 34s) [production]
02:25 <mwdeploy@tin> sync-l10n completed (1.27.0-wmf.13) (duration: 12m 28s) [production]
2016-02-15 §
22:54 <andrewbogott> Andrew encourages future puppet hackers to commit their local changes so that this mess doesn’t reappear [phabricator]
22:52 <andrewbogott> stashed local changes in /var/lib/git/operations/puppet on deploy instance, checked out ‘production’ branch, updated [phabricator]
22:32 <andrewbogott> rebooting instance phab-02 [phabricator]
22:29 <andrewbogott> rebooting instance phab-01 [phabricator]
21:26 <krinkle@tin> Synchronized w/static.php: (no message) (duration: 00m 58s) [production]
20:55 <twentyafterfour> moved contents of /srv/phab/dumps into /srv/dumps [production]
20:48 <twentyafterfour> backing up /srv/phab to /srv/phab.bak in case it needs to be restored in a hurry. Note: the backup excludes /srv/phab/repos which is backed up separately [production]
20:47 <twentyafterfour> brought iridium up to date with a current puppet run and checked that all repositories are at the correct tag [production]
20:27 <bd808> Updated to 3314de8; Guard against %D\d matching Pabricator echo; De-dup Phabricator echos [tools.stashbot]
18:23 <volans> Upgrading db1022 (MariaDB and kernel) already depooled and put in scheduled downtime [production]
16:46 <yurik> Deployed patch for T126897 [production]
16:38 <volans@tin> Synchronized wmf-config/db-eqiad.php: depool db1022 (duration: 00m 58s) [production]
16:34 <bd808> Ran sync-common on mw1119 after stuck scap update was killed there [production]
16:29 <bd808> Killed stuck rsync by mw1119 that was keeping l10nudpate running [production]
16:28 <mwdeploy@tin> sync-l10n completed (1.27.0-wmf.13) (duration: 855m 18s) [production]
16:28 <jzerebecki> reloading zuul for 2d16ad3..3bb0afa [releng]
16:27 <moritzm> installing postgres security updates on maps* [production]
16:26 <halfak> restarted workers on ores-worker -02 and -03 [ores]
16:10 <hashar> Image ci-jessie-wikimedia-1455552377 in wmflabs-eqiad is ready [releng]
15:32 <_joe_> restarting the salt minions on all deployment targets [production]
15:25 <jzerebecki> reloading zuul for e174335..2d16ad3 [releng]
15:23 <hashar> Image ci-jessie-wikimedia-1455549539 in wmflabs-eqiad is ready [releng]
15:22 <apergos> disabled puppet again on iridium, had to restore sshd config from filebucket after puppet run [production]
15:19 <hashar> Regenerating Nodepool snapshot. Slave scripts have 0 bytes... [releng]
15:18 <hoo|busy> Changed email for global account "Frau pomerenke" [production]
15:04 <hashar> Slave scripts added to Nodepool instances! Image ci-jessie-wikimedia-1455548346 in wmflabs-eqiad is ready [releng]
14:51 <apergos> phab role commented out on iridium and puppet re-enabled, no phab changes will be applied there [production]
14:24 <godog> start restbase1007-b cassandra instance, bootstrapping T119935 [production]
13:22 <joal> Failed deploying refinery: deploy from tin failed, need ottomata help [analytics]
13:04 <joal> Deploying refinery [analytics]
12:30 <_joe_> deployment master is now tin [production]
11:57 <_joe_> switching the deployment host back to tin [production]
11:48 <_joe_> restarted hhvm on mw1130, memory exhausted [production]
11:28 <jynus> offlining disk 32:5 on db1063 [production]
11:05 <hashar> Image ci-jessie-wikimedia-1455534001 in wmflabs-eqiad is ready [releng]
10:58 <hoo@mira> Synchronized wmf-config/throttle.php: Throttle exception for [[de:WP:Wikimedia Deutschland/WPFF Berlinale2016]] (duration: 01m 28s) [production]
10:57 <hashar> bumped Nodepool disk image to fix apt configuration / include jre and package-config. Refreshing snapshot image. Poke T126246 T126230 T119143 [releng]
09:38 <godog> swift codfw-prod: ms-be2020 / ms-be2021 weight to 3500 [production]
09:16 <gehel> re-enabling puppet on elastic05 (https://phabricator.wikimedia.org/T126891) [deployment-prep]
09:09 <_joe_> powercycled mw1131, OOM'd [production]
07:52 <legoktm> deploying https://gerrit.wikimedia.org/r/270686 [releng]
06:52 <legoktm> legoktm@gallium:/srv/org/wikimedia/doc$ sudo -u jenkins-slave rm -rf EventLogging/ GuidedTour/ MultimediaViewer/ TemplateData/ [releng]