|
2015-07-23
ยง
|
| 17:38 |
<legoktm> |
running foreachwikiindblist /home/legoktm/largebutnotenwiki.dblist populateContentModel.php --ns=all --table=page |
[production] |
| 16:38 |
<marxarelli> |
Reloading Zuul to deploy I96b6218a208f133209452c71bcf01a1088305aea |
[releng] |
| 16:27 |
<ori> |
restarted hhvm on mw1221 |
[production] |
| 16:16 |
<thcipriani> |
Finished scap: SWAT: Add azb interwiki sorting, Add Southern Luri, and Fix name of S and W Balochi (duration: 06m 13s) |
[production] |
| 16:14 |
<urandom> |
restarting Cassandra on restbase1001 to (temporarily) enable GC logging |
[production] |
| 16:10 |
<thcipriani> |
Started scap: SWAT: Add azb interwiki sorting, Add Southern Luri, and Fix name of S and W Balochi |
[production] |
| 15:40 |
<urandom> |
applied wip logstash & cassandra changes (https://gerrit.wikimedia.org/r/#/c/226025/) to deployment-prep |
[releng] |
| 15:38 |
<moritzm> |
added jenkins-debian-glue 0.13.0 to apt.wikimedia.org (jessie-wikimedia) |
[production] |
| 15:35 |
<thcipriani> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: fix references to non-existent wikis [[gerrit:226470]] (duration: 00m 13s) |
[production] |
| 15:31 |
<_joe_> |
rebooting ms-be1003, stuck in kernel locks |
[production] |
| 15:31 |
<thcipriani> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: Remove reference to nonexistent ru_sibwiki.png [[gerrit:226469]] (duration: 00m 14s) |
[production] |
| 15:26 |
<thcipriani> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: Add wgSitename and wgMetaNamespace for pnbwiki [[gerrit:226543]] (duration: 00m 12s) |
[production] |
| 15:15 |
<thcipriani> |
Synchronized wmf-config/CommonSettings.php: SWAT: Set a different wmgContentTranslationDefaultSourceLanguage for English part II [[gerrit:224031]] (duration: 00m 12s) |
[production] |
| 15:14 |
<thcipriani> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: Set a different wmgContentTranslationDefaultSourceLanguage for English part I [[gerrit:224031]] (duration: 00m 13s) |
[production] |
| 15:04 |
<thcipriani> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: Add wgSitename and wgMetaNamespace for pnbwikipedia [[gerrit:225322]] (duration: 00m 12s) |
[production] |
| 13:24 |
<hashar> |
apt-get upgrade integration-puppetmaster and rebooting it |
[releng] |
| 13:23 |
<hashar> |
integration puppetmaster in bad shape: Warning: Error 400 on SERVER: Cannot allocate memory - fork(2) |
[releng] |
| 13:08 |
<mobrovac> |
graphoid deploying 81b9633 |
[production] |
| 10:58 |
<hashar> |
beta : salt '*' cmd.run 'rm /etc/apt/apt.conf.d/20auto-upgrades.ucf-dist' |
[releng] |
| 10:57 |
<jynus> |
disabling puppet on maps-test hosts to debug service issue |
[production] |
| 10:52 |
<hashar> |
Beta cluster puppetmaster is now deployment-puppetmaster.deployment-prep.eqiad.wmflabs . Migrated all instances (solves https://phabricator.wikimedia.org/T106649 ) |
[releng] |
| 10:30 |
<hashar> |
regenerated puppet cert on deployment-salt , the old puppetmaster now a puppet client |
[releng] |
| 10:23 |
<hashar> |
running apt-get upgrade on deployment-parsoidcache02 |
[releng] |
| 09:32 |
<hashar> |
puppet broken on deployment-fluorine : Error: Could not request certificate: Neither PUB key nor PRIV key:: header too long |
[releng] |
| 08:39 |
<hashar> |
Disabling puppet agent on ALL beta cluster instances |
[releng] |
| 08:18 |
<hashar> |
creating deployment-puppetmaster m1.medium :D |
[releng] |
| 07:28 |
<_joe_> |
upgrading hhvm on the canary appservers |
[production] |
| 06:59 |
<LocalisationUpdate> |
ResourceLoader cache refresh completed at Thu Jul 23 06:59:44 UTC 2015 (duration 59m 43s) |
[production] |
| 06:42 |
<springle> |
Synchronized wmf-config/db-eqiad.php: repool db1070, warm up (duration: 00m 13s) |
[production] |
| 04:25 |
<ori> |
Synchronized php-1.26wmf15/extensions/Scribunto/common/Base.php: (no message) (duration: 00m 13s) |
[production] |
| 04:24 |
<ori> |
Synchronized php-1.26wmf14/extensions/Scribunto/common/Base.php: (no message) (duration: 00m 12s) |
[production] |
| 04:04 |
<springle> |
upgrade & reboot db1070 |
[production] |
| 03:04 |
<LocalisationUpdate> |
completed (1.26wmf15) at 2015-07-23 03:04:48+00:00 |
[production] |
| 03:00 |
<l10nupdate> |
Synchronized php-1.26wmf15/cache/l10n: (no message) (duration: 07m 24s) |
[production] |
| 02:39 |
<springle> |
temporarily silenced backup4001 check_disk space icinga noise; seems important, but not exploding-any-minute-now |
[production] |
| 02:38 |
<LocalisationUpdate> |
completed (1.26wmf14) at 2015-07-23 02:37:55+00:00 |
[production] |
| 02:34 |
<l10nupdate> |
Synchronized php-1.26wmf14/cache/l10n: (no message) (duration: 07m 13s) |
[production] |
| 02:07 |
<LocalisationUpdate> |
ResourceLoader cache refresh completed at Thu Jul 23 02:07:12 UTC 2015 (duration 7m 11s) |
[production] |
| 02:05 |
<springle> |
Synchronized wmf-config/db-eqiad.php: depool db1070 (duration: 00m 12s) |
[production] |
| 02:03 |
<LocalisationUpdate> |
failed (1.26wmf15) at 2015-07-23 02:03:03+00:00 |
[production] |
| 02:03 |
<LocalisationUpdate> |
failed (1.26wmf14) at 2015-07-23 02:03:02+00:00 |
[production] |
| 01:57 |
<jzerebecki> |
reconnected slave and needed to kill a few pending beta jobs, works again |
[releng] |
| 01:50 |
<jzerebecki> |
trying https://www.mediawiki.org/wiki/Continuous_integration/Jenkins#Hung_beta_code.2Fdb_update |
[releng] |
| 01:45 |
<ori> |
Synchronized php-1.26wmf15/includes/libs/objectcache/APCBagOStuff.php: I4b2cf1715538 (duration: 00m 12s) |
[production] |
| 01:45 |
<ori> |
Synchronized php-1.26wmf14/includes/libs/objectcache/APCBagOStuff.php: I4b2cf1715538 (duration: 00m 12s) |
[production] |
| 01:09 |
<legoktm> |
beta-mediawiki-config-update-eqiad jobs stuck |
[releng] |
| 01:05 |
<twentyafterfour> |
phab is back |
[production] |
| 01:04 |
<ori> |
Synchronized php-1.26wmf14/includes/libs/objectcache/APCBagOStuff.php: I4b2cf1715 (duration: 00m 12s) |
[production] |
| 01:01 |
<legoktm> |
twentyafterfour is upgrading phabricator |
[production] |
| 00:50 |
<yurik> |
deployed kartotherian fix, still not starting as a service, and no idea why. Have no access to logs. Frustrated. |
[production] |