|
2014-03-20
§
|
| 21:43 |
<ori> |
updated /a/common to {{Gerrit|If51eda243}}: Follow up Id6222f4db to amend sort order in feed URL |
[production] |
| 21:30 |
<hashar> |
manually installing timidity-daemon on jobrunner01.eqiad so puppet can stop it and stop whining |
[releng] |
| 21:20 |
<ori> |
synchronized wmf-config/InitialiseSettings.php 'Id6222f4d: Add RSS of Bugzilla query of open HHVM bugs to mediawikiwiki's whitelist' |
[production] |
| 21:20 |
<ori> |
updated /a/common to {{Gerrit|Id6222f4db}}: Add RSS of Bugzilla query of open HHVM bugs to mediawikiwiki's whitelist |
[production] |
| 21:06 |
<reedy> |
synchronized docroot and w |
[production] |
| 21:03 |
<reedy> |
updated /a/common to {{Gerrit|Iaa99d2162}}: Add Wikibase repoSiteName setting for client |
[production] |
| 21:00 |
<hashar> |
migrate jobrunner01.eqiad.wmflabs to self puppet/salt masters |
[releng] |
| 20:56 |
<bd808> |
Updated scholarships.wikimedia.org to cb2ef4c (fix for bug 62464) |
[production] |
| 20:55 |
<hashar> |
deleting deployment-jobrunner02 , lets start with a single instance for nwo |
[releng] |
| 20:51 |
<hashar> |
Creating deployment-jobrunner01 and 02 in eqiad. |
[releng] |
| 20:14 |
<mutante> |
DNS update - remove ssl1-4 |
[production] |
| 20:08 |
<mutante> |
DNS update - remove sq67-70, former varnish testing |
[production] |
| 19:35 |
<akosiaris> |
created a 50G LV for /var/log on zirconium, stopped all services, moved data to it, mounted it and restarted all services |
[production] |
| 19:29 |
<reedy> |
synchronized wmf-config/ 'Wikibase config updates' |
[production] |
| 19:23 |
<reedy> |
Finished scap: Rebuild 1.23wmf19 l10n cache for wikibase (duration: 12m 01s) |
[production] |
| 19:10 |
<reedy> |
Started scap: Rebuild 1.23wmf19 l10n cache for wikibase |
[production] |
| 19:08 |
<reedy> |
rebuilt wikiversions.cdb and synchronized wikiversions files: group0 wikis to 1.23wmf19 |
[production] |
| 19:02 |
<reedy> |
rebuilt wikiversions.cdb and synchronized wikiversions files: Wikipedias to 1.23wmf18 |
[production] |
| 19:00 |
<mutante> |
disk full on zirconium - gzipping some etherpadlite.sql dump i found |
[production] |
| 17:41 |
<reedy> |
synchronized php-1.23wmf19 'Update Wikidata and WikimediaMessages' |
[production] |
| 17:06 |
<Krinkle> |
Reloading Zuul to deploy Ie800ed90b51c47d5a1 |
[production] |
| 16:58 |
<mutante> |
repooling mw1163 (it's back in dsh as well) |
[production] |
| 16:38 |
<reedy> |
rebuilt wikiversions.cdb and synchronized wikiversions files: testwiki back to 1.23wmf18 till window |
[production] |
| 16:05 |
<bd808> |
Ran /usr/local/bin/sync-common && /usr/local/bin/scap-rebuild-cdbs on mw1163. Should not repool until it's back in the dsh group. Should me manually synced just before repooling. |
[production] |
| 15:47 |
<hashar> |
fixed salt-minion service on deployment-cache-upload01 and deployment-cache-mobile03 by deleting /etc/salt/pki/minion/minion_master.pub |
[releng] |
| 15:36 |
<reedy> |
Finished scap: testwiki to 1.23wmf19 and build l10n cache (duration: 15m 32s) |
[production] |
| 15:30 |
<hashar> |
migrated deployment-cache-upload01.eqiad.wmflabs and deployment-cache-mobile03.eqiad.wmflabs to use the salt/puppetmaster deployment-salt.eqiad.wmflabs. |
[releng] |
| 15:30 |
<hashar> |
deployment-cache-upload01.eqiad.wmflabs and deployment-cache-mobile03.eqiad.wmflabs recovered!! /dev/vdb does not exist on eqiad which caused the instance to be stalled. |
[releng] |
| 15:21 |
<reedy> |
Started scap: testwiki to 1.23wmf19 and build l10n cache |
[production] |
| 15:03 |
<maxsem> |
synchronized wmf-config/InitialiseSettings.php 'https://gerrit.wikimedia.org/r/118328' |
[production] |
| 15:00 |
<bd808> |
logstash stopped ingesting logs at 2014-03-19T22:37:54.000Z. |
[production] |
| 14:57 |
<bd808> |
restarted logstash on logstash1001.eqiad.wmnet |
[production] |
| 10:48 |
<hashar> |
Stopped the simplewiki script. Would need to recreate the db from scratch instead |
[releng] |
| 10:37 |
<hashar> |
Cleaning up simplewiki by deleting most pages in the main namespace. Would free up some disk space. deleteBatch.php is running in a screen on deployment-bastion.pmtpa.wmflabs |
[releng] |
| 10:08 |
<hashar> |
applying role::labs::lvm::mnt on deployment-db1 to provide additional disk space on /mnt |
[releng] |
| 09:39 |
<hashar> |
convert all remaining hosts but db1 to use the local puppet and salt masters |
[releng] |
| 03:23 |
<LocalisationUpdate> |
ResourceLoader cache refresh completed at Thu Mar 20 03:23:37 UTC 2014 (duration 23m 35s) |
[production] |
| 02:32 |
<LocalisationUpdate> |
completed (1.23wmf18) at 2014-03-20 02:32:10+00:00 |
[production] |
| 02:17 |
<LocalisationUpdate> |
completed (1.23wmf17) at 2014-03-20 02:17:34+00:00 |
[production] |
| 01:26 |
<Tim> |
gave Sam Reed access to racktables |
[production] |
| 00:26 |
<bd808> |
scap-slave rsync servers have "hosts allow = 10.0.0.0/16 10.64.0.0/22 10.64.16.0/22 10.64.32.0/22 208.80.152.0/22"; missing new 10.64.48.0/22 subnet |
[production] |
| 00:21 |
<bd808> |
Subnet for row D still not in rsync server config? |
[production] |
| 00:21 |
<bd808> |
scaps failed because "@ERROR: access denied to common from mw1202.eqiad.wmnet (10.64.48.34)" |
[production] |
| 00:16 |
<bd808> |
finished dsh -c -M -m mw1201,mw1202,mw1203,mw1208,mw1209,mw1210 -- '/usr/local/bin/sync-common; /usr/local/bin/scap-rebuild-cdbs' |
[production] |
| 00:13 |
<bd808> |
dsh -c -M -m mw1201,mw1202,mw1203,mw1208,mw1209,mw1210 -- '/usr/local/bin/sync-common; /usr/local/bin/scap-rebuild-cdbs' |
[production] |