|
2015-01-07
§
|
| 18:42 |
<godog> |
reboot ms-be2011, megacli in a funny state and unable to bring new drive in service |
[production] |
| 17:20 |
<ori> |
Synchronized php-1.25wmf13/extensions/EventLogging/modules/ext.eventLogging.core.js: I5470424: Correct events to send schema name (duration: 00m 05s) |
[production] |
| 17:20 |
<ori> |
Synchronized php-1.25wmf12/extensions/EventLogging/modules/ext.eventLogging.core.js: I5470424: Correct events to send schema name (duration: 00m 06s) |
[production] |
| 17:15 |
<godog> |
reboot ms-be2003 |
[production] |
| 16:25 |
<YuviPanda> |
added milimetric to NDA sudo’ers groups |
[releng] |
| 15:37 |
<springle> |
xtrabackup clone codfw slaves db2034 db2035 db2036 db2037 db2038 db2039 db2040 from other codfw slaves |
[production] |
| 14:40 |
<_joe_> |
upgrading hhvm on testwiki |
[production] |
| 14:16 |
<springle> |
xtrabackup clone db1027 to db1015 |
[production] |
| 11:42 |
<_joe_> |
reimaging mw1009-mw1012 |
[production] |
| 10:57 |
<hashar> |
Taught Jenkins configuration about Java 8. Name: "Ubuntu - OpenJdk 8" JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64/ . Only available on Trusty slaves though |
[releng] |
| 10:56 |
<hashar> |
installed openjdk 8 on CI Trusty labs slaves https://phabricator.wikimedia.org/T85964 |
[releng] |
| 10:34 |
<hashar> |
varnish text cache is back up. Had to delete /etc/varnish and reinstall varnish from scratch + rerun puppet. |
[releng] |
| 10:25 |
<hashar> |
deleting /etc/varnish on deplloyment-cache-text02 and running puppet |
[releng] |
| 10:24 |
<hashar> |
beta varnish text cache is broken. The vcl refuses to load because of undefined probes |
[releng] |
| 10:19 |
<godog> |
reboot ms-be2003, deleted LD should disappear |
[production] |
| 10:01 |
<hashar> |
restarted deployment-cache-mobile03 and deployment-cache-text02 |
[releng] |
| 09:50 |
<hashar> |
rebooting deployment-cache-bits01 |
[releng] |
| 09:47 |
<hashar> |
restarting Jenkins to resolve a deadlocks with the beta cluster jobs |
[production] |
| 07:53 |
<_joe_> |
reimaging jobrunners mw1013-mw1016 (in batch of two) |
[production] |
| 06:50 |
<springle> |
xtrabackup clone es2008 to es2010 |
[production] |
| 06:50 |
<springle> |
xtrabackup clone es2006 to es2007 |
[production] |
| 02:05 |
<kaldari> |
Synchronized php-1.25wmf13/extensions/WikiGrok/: Fixing campaign generation in WikiGrok (duration: 00m 05s) |
[production] |
| 01:57 |
<ori> |
Synchronized php-1.25wmf13/extensions/EventLogging: I69c8daf: Update EventLogging for cherry-picks (duration: 00m 05s) |
[production] |
| 01:57 |
<ori> |
Synchronized php-1.25wmf12/extensions/EventLogging: I07d9bc8: Update EventLogging for cherry-picks (duration: 00m 06s) |
[production] |
| 01:33 |
<springle> |
xtrabackup clone es2008 to es2009 |
[production] |
| 01:24 |
<kaldari> |
Synchronized php-1.25wmf13/extensions/MobileFrontend/: Sync MobileFrontend in 1.25wmf13 for VE fix (duration: 00m 05s) |
[production] |
| 01:11 |
<springle> |
xtrabackup clone es2006 to es2005 |
[production] |
| 00:41 |
<Krinkle> |
rm -rf slave-scripts and re-cloning from integration/jenkins.git on all slaves (under sudo, just like puppet originally did) - git-status and jshint both work fine now |
[releng] |
| 00:40 |
<Krinkle> |
Permissions of deployment/integration/slave-scripts on labs slave are all screwed up (git-status says files are dirty, but when run as root git-status is clean and jshint also works fine via sudo) |
[releng] |
| 00:29 |
<Krinkle> |
Tried reconnecting Gearman, relaunching slave agents. Force-restarting Zuul now. |
[releng] |
| 00:15 |
<Krinkle> |
Permissions in deployment/integration/slave-scripts on integration-slave1003 are screwed up as well |
[releng] |
|
2015-01-06
§
|
| 22:13 |
<hashar> |
jshint complains with: Error: Cannot find module './lib/node' :-( |
[releng] |
| 22:12 |
<hashar> |
integration-slave1005 chmod -R go+r /srv/deployment/integration/slave-scripts |
[releng] |
| 22:08 |
<hashar> |
integration-slave1007 chmod -R go+r /srv/deployment/integration/slave-scripts . cscott mentioned build failures of parsoidsvc-jslint which could not read /srv/deployment/integration/slave-scripts/tools/node_modules/jshint/src/cli.js |
[releng] |
| 21:49 |
<awight> |
update crm from 80241fd2a43f03796b416d728661470f875a590a to 8ded33737337d0c842d8c194b5cb15b25fc99c2e |
[production] |
| 20:24 |
<reedy> |
Synchronized wmf-config/: (no message) (duration: 00m 06s) |
[production] |
| 20:16 |
<Reedy> |
Ran namespaceDupes.php on ndswiktionary |
[production] |
| 20:16 |
<reedy> |
Synchronized wmf-config/: Config updates (duration: 00m 06s) |
[production] |
| 20:00 |
<reedy> |
Synchronized docroot and w: update noc index (duration: 00m 06s) |
[production] |
| 19:42 |
<demon> |
Finished scap: mwsearch is no more (duration: 35m 40s) |
[production] |
| 19:06 |
<demon> |
Started scap: mwsearch is no more |
[production] |
| 19:03 |
<reedy> |
rebuilt wikiversions.cdb and synchronized wikiversions files: nooop |
[production] |
| 19:01 |
<reedy> |
rebuilt wikiversions.cdb and synchronized wikiversions files: Non wikipedias to 1.25wmf13 |
[production] |
| 18:30 |
<godog> |
upload txstatsd 1.0.0-2 to trusty-wikimedia |
[production] |
| 16:22 |
<anomie> |
Synchronized php-1.25wmf13/extensions/VisualEditor/modules/ve-mw/init/targets/ve.init.mw.MobileViewTarget.js: SWAT: Update setupToolbar signature in mobile target [[gerrit:182993]] (duration: 00m 06s) |
[production] |
| 16:01 |
<anomie> |
Synchronized wmf-config/flaggedrevs.php: SWAT: Add Flexion to $wgFlaggedRevsNamespaces for dewiktionary [[gerrit:180813]] (duration: 00m 08s) |
[production] |
| 10:36 |
<hashar> |
The Zuul issues have their root cause in Gerrit which cannot open ReviewDB. Filled as https://phabricator.wikimedia.org/T85916 |
[production] |
| 10:21 |
<hashar> |
Restarted Zuul. Gerrit transiently died out just like ~10 hours ago which locked Zuul entirely |
[production] |
| 09:41 |
<springle> |
started an analytics ETL run on dbstore1002. to disable: set global event_scheduler=0; |
[production] |
| 09:41 |
<godog> |
starting hhvm-profiler-to-carbon on tungsten T85641 |
[production] |