|
2013-11-22
§
|
| 09:20 |
<apergos> |
snmptt stuck waiting for a submit_check_result that had exited (or died), restarted it to clear the snmptt spool queue |
[production] |
| 08:14 |
<apergos> |
rebooting neon again, found four filenames with deleted inode but side effect is back in r/o for / |
[production] |
| 07:06 |
<apergos> |
rebooting neon, no underlying errors found except for the initial ext3 'deleted inode reference' which caused / to be remounted r/o |
[production] |
| 06:53 |
<apergos> |
neon back to read-only root filesystem, investigating |
[production] |
| 03:23 |
<LocalisationUpdate> |
ResourceLoader cache refresh completed at Fri Nov 22 03:23:33 UTC 2013 |
[production] |
| 02:35 |
<LocalisationUpdate> |
completed (1.23wmf5) at Fri Nov 22 02:35:51 UTC 2013 |
[production] |
| 02:16 |
<LocalisationUpdate> |
completed (1.23wmf4) at Fri Nov 22 02:16:36 UTC 2013 |
[production] |
| 01:41 |
<csteipp> |
synchronized wmf-config/InitialiseSettings.php |
[production] |
| 01:40 |
<csteipp> |
synchronized wmf-config/CommonSettings.php |
[production] |
| 01:37 |
<csteipp> |
finished scap: Really send the new messages this time.. |
[production] |
| 01:16 |
<csteipp> |
started scap: Really send the new messages this time.. |
[production] |
| 00:56 |
<ori> |
synchronized php-1.23wmf4/resources/startup.js 'touch' |
[production] |
| 00:50 |
<csteipp> |
finished scap: Updating WikimediaMessages to master for OAuth message |
[production] |
| 00:50 |
<ori> |
synchronized php-1.23wmf4/extensions/MobileFrontend/javascripts/loggingSchemas/MobileWebInfobox.js 'Cherry-pick I3efc1fa64' |
[production] |
| 00:46 |
<ori> |
synchronized php-1.23wmf4/resources/startup.js 'touch' |
[production] |
| 00:36 |
<csteipp> |
started scap: Updating WikimediaMessages to master for OAuth message |
[production] |
| 00:35 |
<ori> |
synchronized php-1.23wmf4/extensions/MobileFrontend/javascripts/loggingSchemas/MobileWebInfobox.js 'Cherry-pick I3efc1fa64' |
[production] |
| 00:09 |
<demon> |
synchronized wmf-config/PoolCounterSettings-eqiad.php 'New pool counter settings for Cirrus' |
[production] |
| 00:08 |
<demon> |
synchronized wmf-config/PoolCounterSettings-pmtpa.php 'New pool counter settings for Cirrus' |
[production] |
| 00:07 |
<Reedy> |
Built texvc for 1.23wmf5 |
[production] |
|
2013-11-21
§
|
| 23:34 |
<ori-l> |
Purging tifs older than one hour on app servers |
[production] |
| 23:06 |
<reedy> |
synchronized wmf-config/InitialiseSettings.php 'Enable BetaFeatures everywher' |
[production] |
| 23:03 |
<reedy> |
updated /a/common to {{Gerrit|I3ac8621d9}}: Removed ZERO namespace protection in META |
[production] |
| 23:02 |
<reedy> |
synchronized wmf-config 'last few config updates' |
[production] |
| 22:32 |
<reedy> |
rebuilt wikiversions.cdb and synchronized wikiversions files: phase1 wikis to 1.23wmf5 |
[production] |
| 22:31 |
<reedy> |
synchronized php-1.23wmf5/extensions/MobileFrontend/ |
[production] |
| 22:21 |
<reedy> |
rebuilt wikiversions.cdb and synchronized wikiversions files: Wikipedias to 1.23wmf4 |
[production] |
| 21:47 |
<reedy> |
synchronized wmf-config/InitialiseSettings.php 'rv' |
[production] |
| 21:46 |
<reedy> |
synchronized wmf-config/InitialiseSettings.php 'wiki -' |
[production] |
| 21:26 |
<reedy> |
updated /a/common to {{Gerrit|I4360e8a34}}: Enable BetaFeatures everywhere except Wikipedias |
[production] |
| 21:18 |
<reedy> |
synchronized wmf-config/InitialiseSettings.php 'Enable BetaFeatures on all non wikipedias' |
[production] |
| 21:16 |
<reedy> |
synchronized php-1.23wmf5/extensions/Wikibase |
[production] |
| 21:00 |
<reedy> |
updated /a/common to {{Gerrit|I6433f2b0d}}: testwiki to 1.23wmf5 |
[production] |
| 20:57 |
<cmjohnson1> |
sq48 powering down for decom |
[production] |
| 20:47 |
<cmjohnson> |
sq44 powering down for decom |
[production] |
| 20:24 |
<Reedy> |
mw117 is also segfaulting quite a bit |
[production] |
| 19:56 |
<paravoid> |
depooling mw1183 |
[production] |
| 19:56 |
<Reedy> |
mw1183/10.64.32.53 is having masses of segfaults |
[production] |
| 19:52 |
<Reedy> |
Created BetaFeatures extension tables on all wikis |
[production] |
| 19:46 |
<ori> |
synchronized docroot/noc 'I7ea8302f2: noc.wm.o: add images referenced by stylesheets.' |
[production] |
| 19:45 |
<ori> |
updated /a/common to {{Gerrit|I7ea8302f2}}: noc.wm.o: add images referenced by stylesheets. |
[production] |
| 19:43 |
<reedy> |
Finished syncing Wikimedia installation... : testwiki to 1.23wmf5 and build l10n cache |
[production] |
| 19:33 |
<ori> |
synchronized docroot/noc 'Ia451d47bd: noc.wm.o: use local CSS files rather than point to svn.wm.o' |
[production] |
| 19:32 |
<ori> |
updated /a/common to {{Gerrit|Ia451d47bd}}: noc.wm.o: use local CSS files rather than point to svn.wm.o |
[production] |
| 19:27 |
<reedy> |
Started syncing Wikimedia installation... : testwiki to 1.23wmf5 and build l10n cache |
[production] |
| 19:24 |
<reedy> |
synchronized docroot and w |
[production] |
| 19:21 |
<reedy> |
synchronized php-1.23wmf5 'Staging code' |
[production] |
| 19:02 |
<paravoid> |
restored service for svn.wm.org |
[production] |
| 18:48 |
<cmjohnson> |
shutting down vanadium to replace failed hard drive |
[production] |
| 17:19 |
<^d> |
reloaded zuul to pick up Ibc354db7 |
[production] |