|
2015-12-08
§
|
| 14:18 |
<Krinkle> |
Removed integration-slave-trusty-1012:/mnt/home/jenkins-deploy/tmpfs/jenkins-2 which was left behind by a job. Caused other jobs to fail due to lack of permission to chmod/rm-rf this dir. |
[releng] |
| 14:16 |
<mobrovac> |
restbase canary deploy of f47405a onto restbase1001 |
[production] |
| 14:09 |
<mobrovac> |
restbase running puppet on rb1001 |
[production] |
| 11:48 |
<hashar> |
beta: salt-key --delete deployment-cxserver03.eqiad.wmflabs |
[releng] |
| 11:42 |
<hashar> |
running puppet on deployment-restbase01 . Catch up on lot of changes |
[releng] |
| 11:23 |
<hashar> |
puppet catching up a lot of changes on deployment-cache-mobile04 and deployment-cache-text04 |
[releng] |
| 11:20 |
<hashar> |
beta: rebased puppet.git on puppetmaster |
[releng] |
| 11:02 |
<godog> |
upgrade diamond to 3.5-5 in codfw |
[production] |
| 10:58 |
<hashar> |
dropped deployment-cache-text04 puppet SSL certificates |
[releng] |
| 10:44 |
<hashar> |
beta: deployment-cache-text04 upgrading openssl libssl1.0.0 |
[releng] |
| 10:42 |
<hashar> |
beta: fixing salt on bunch of hosts. There are duplicate process on a few of them. Fix up is: killall salt-minion && rm /var/run/salt-minion.pid && /etc/init.d/salt-minion start |
[releng] |
| 10:32 |
<hashar> |
beta: salt-key --delete=deployment-cache-upload04.eqiad.wmflabs (missing 'deployment-prep' subdomain) |
[releng] |
| 10:31 |
<hashar> |
beta: puppet being fixed on memc04 sentry2 cache-upload04 cxserver03 db1 |
[releng] |
| 10:27 |
<hashar> |
beta: fixing puppet.con on a bunch of hosts. The [agent] server = deployment-puppetmaster.eqiad.wmflabs is wrong, missing 'deployment-prep' sub domain |
[releng] |
| 10:23 |
<hashar> |
beta: salt-key --delete=i-000005d2.eqiad.wmflabs |
[releng] |
| 10:04 |
<godog> |
start cassandra on restbase1008, bootstrapping |
[production] |
| 09:56 |
<hashar> |
gallium: upgraded python-diamond and java. Restarting Jenkins |
[production] |
| 09:48 |
<hashar> |
gallium: removing bunch of :i386 packages which were used for Android build but got removed ( https://gerrit.wikimedia.org/r/#/c/183790/3/modules/androidsdk/manifests/dependencies.pp,unified ) |
[production] |
| 09:15 |
<godog> |
reimage restbase1008 |
[production] |
| 05:45 |
<gwicke> |
restbase1008: disable cassandra with `systemctl mask cassandra` |
[production] |
| 04:00 |
<l10nupdate@tin> |
ResourceLoader cache refresh completed at Tue Dec 8 04:00:02 UTC 2015 (duration 1h 34m 44s) |
[production] |
| 03:12 |
<gwicke> |
Keeping puppet on restbase production cluster disabled until Marko & Filippo can deploy tomorrow morning. PLEASE DO NOT ENABLE! |
[production] |
| 02:25 |
<mwdeploy@tin> |
sync-l10n completed (1.27.0-wmf.7) (duration: 09m 56s) |
[production] |
| 01:16 |
<catrope@tin> |
Synchronized wmf-config/InitialiseSettings.php: Also blacklist Minerva for WPB (duration: 00m 28s) |
[production] |
| 00:59 |
<gwicke> |
disabling puppet in production restbase cluster in preparation for testing https://gerrit.wikimedia.org/r/#/c/257408/ in staging |
[production] |
| 00:39 |
<catrope@tin> |
Synchronized php-1.27.0-wmf.7/extensions/MobileFrontend/: SWAT (duration: 00m 28s) |
[production] |
| 00:38 |
<catrope@tin> |
Synchronized php-1.27.0-wmf.7/extensions/MwEmbedSupport/: SWAT (duration: 00m 28s) |
[production] |
| 00:37 |
<catrope@tin> |
Synchronized php-1.27.0-wmf.7/resources/src/oojs-ui-local.js: SWAT: connect OOjs UI to MW's l10n system (duration: 00m 28s) |
[production] |
| 00:37 |
<catrope@tin> |
Synchronized php-1.27.0-wmf.7/resources/ResourcesOOUI.php: SWAT: connect OOjs UI to MW's l10n system (duration: 00m 27s) |
[production] |
| 00:34 |
<catrope@tin> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: enable banners on mobile web beta on enwiki; remove MobileFrontend Browse config (duration: 00m 27s) |
[production] |
| 00:31 |
<catrope@tin> |
Synchronized wmf-config/squid.php: SWAT: stop using separate address for upload in wgHTCPRouting (duration: 00m 29s) |
[production] |
| 00:07 |
<catrope@tin> |
Synchronized wmf-config/InitialiseSettings.php: Enable Flow opt-in beta feature on bswiki and urwiki (duration: 00m 28s) |
[production] |
|
2015-12-07
§
|
| 22:29 |
<ori@tin> |
Synchronized php-1.27.0-wmf.7/extensions/MobileFrontend: I8c94425693: Improve disableImages cookie code (T120151) (duration: 00m 30s) |
[production] |
| 22:22 |
<ebernhardson@tin> |
Synchronized portals/prod: Deploy updated portals (duration: 00m 27s) |
[production] |
| 22:01 |
<mdholloway> |
mobileapps deployed 984f14b |
[production] |
| 21:31 |
<hoo@tin> |
Synchronized php-1.27.0-wmf.7/extensions/Wikidata/: (no message) (duration: 00m 37s) |
[production] |
| 21:31 |
<ottomata> |
temporarily stopping eventlogging and mysql in beta |
[analytics] |
| 21:15 |
<subbu> |
finished deploying parsoid sha 4a7df427 (cf0b9ef + cherry-pick of d65debd) |
[production] |
| 21:09 |
<subbu> |
restart parsoid on wtp1002 as a canary |
[production] |
| 21:05 |
<subbu> |
starting parsoid deploy |
[production] |
| 20:39 |
<valhallasw`cloud> |
wb2-irc thinks it's connected but messages don't actually get out to IRC. Restarting. |
[tools.wikibugs] |
| 20:33 |
<andrewbogott> |
enabling puppet on labcontrol1002, testing over |
[production] |
| 20:07 |
<andrewbogott> |
disabling puppet on labcontrol1002 for an ldap/pdns test |
[production] |
| 19:52 |
<jynus@tin> |
Synchronized wmf-config/db-eqiad.php: Repool es1015 at 100% load (duration: 00m 27s) |
[production] |
| 19:52 |
<andrewbogott> |
restarting nova-compute on all labvirt1X nodes |
[production] |
| 19:20 |
<andrewbogott> |
restarted pdns on labcontrol1001 and 1002 to catch up with recent opendj restarts |
[production] |
| 19:11 |
<andrewbogott> |
restarting ldap on neptunium and nembus due to replication warnings in the logs |
[production] |
| 18:36 |
<jynus> |
es1019 and its management interface became unresponsive |
[production] |
| 18:16 |
<YuviPanda> |
it is dead |
[orgcharts] |
| 18:16 |
<mutante> |
deleting pmtpa host groups from nagios/icinga |
[production] |