|
2018-03-14
§
|
| 06:45 |
<marostegui> |
Deploy schema change on db1064 with replication (this will generate lag on s4 on labs hosts) - T187089 T185128 T153182 |
[production] |
| 06:39 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Depool db1064 for alter table (duration: 01m 14s) |
[production] |
| 06:30 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Repool db1097:3314 after alter table (duration: 01m 15s) |
[production] |
| 03:13 |
<mutante> |
bacula is working again - restored missing file set (https://gerrit.wikimedia.org/r/419341 ) |
[production] |
| 02:49 |
<l10nupdate@tin> |
scap sync-l10n completed (1.31.0-wmf.24) (duration: 05m 40s) |
[production] |
| 02:44 |
<Jamesofur> |
deleted 46 archived files |
[production] |
| 02:18 |
<mutante> |
helium - running bacula-dir with -f in foreground revealed: ERROR TERMINATION at parse_conf.c:485 - Config error: Could not find config Resource mysql-srv-backups - line 7, col 33 of file /etc/bacula/jobs.d/bohrium.eqiad.wmnet-mysql-predump-piwik-Weekly-Wed-production.conf |
[production] |
| 02:17 |
<mutante> |
helium - bacula director process failed (Bacula interrupted by signal 11: Segmentation violation), icinga alerted. attempted to restart it. then: bacula-dir - the configtest failed! |
[production] |
| 00:01 |
<reedy@tin> |
Synchronized wmf-config/InitialiseSettings.php: crwiki logo (duration: 01m 15s) |
[production] |
| 00:00 |
<reedy@tin> |
Synchronized static/images/project-logos/crwiki.png: (no justification provided) (duration: 01m 14s) |
[production] |
|
2018-03-13
§
|
| 23:46 |
<reedy@tin> |
Synchronized php-1.31.0-wmf.24/extensions/MobileFrontend/: T188825 (duration: 01m 18s) |
[production] |
| 23:43 |
<mutante> |
tin: chmod -R g+w /srv/mediawiki-staging/.git/objects/* ; chmod -R g+w /srv/mediawiki-staging/php-1.31.0-wmf.24/.git/objects/* |
[production] |
| 23:35 |
<Reedy> |
that was Enable VirtualPageViews on Hungarian Wikipedia T184793 |
[production] |
| 23:35 |
<reedy@tin> |
Synchronized wmf-config/InitialiseSettings.php: (no justification provided) (duration: 01m 15s) |
[production] |
| 23:26 |
<reedy@tin> |
Synchronized wmf-config/InitialiseSettings.php: moar logos (duration: 01m 15s) |
[production] |
| 23:24 |
<reedy@tin> |
Synchronized static/images/project-logos/: YOU GET A LOGO, YOU GET A LOGO. YOU ALL GET LOGOS (duration: 01m 16s) |
[production] |
| 23:11 |
<reedy@tin> |
Synchronized wmf-config/InitialiseSettings.php: Enable RemexHTML on 96 wikis T188010 (duration: 01m 16s) |
[production] |
| 23:10 |
<mutante> |
restbase-dev1006 - reinstalling, manually skipping " Volume group name already in use" (T185494) |
[production] |
| 22:52 |
<eileen> |
civicrm revision changed from c8458c4a2f to 9e79d63426, config revision is 08b7e6216e (Benevity comma fix) |
[production] |
| 20:40 |
<demon@tin> |
rebuilt and synchronized wikiversions files: group0 to wmf.25 |
[production] |
| 20:09 |
<demon@tin> |
Finished scap: bootstrap wmf.25 (duration: 67m 17s) |
[production] |
| 19:02 |
<demon@tin> |
Started scap: bootstrap wmf.25 |
[production] |
| 18:47 |
<demon@tin> |
scap failed: LockFailedError Failed to acquire lock "/var/lock/scap.operations_mediawiki-config.lock"; owner is "awight"; reason is "Beta: Fix ORES thresholds and enable JADE, T181159, T176333" (duration: 00m 00s) |
[production] |
| 18:46 |
<demon@tin> |
scap failed: LockFailedError Failed to acquire lock "/var/lock/scap.operations_mediawiki-config.lock"; owner is "awight"; reason is "Beta: Fix ORES thresholds and enable JADE, T181159, T176333" (duration: 00m 00s) |
[production] |
| 18:42 |
<gehel> |
repool wdqs1004 & wdqs2001 now that data reload is completed T189548 |
[production] |
| 18:39 |
<XenoRyet> |
updated civicrm from 8652db05f5 to c8458c4a2f |
[production] |
| 18:37 |
<moritzm> |
installing reportbug updates from stretch point release |
[production] |
| 18:32 |
<moritzm> |
installing w3m updates from stretch point release |
[production] |
| 17:55 |
<moritzm> |
installing ncurses updates from stretch point release |
[production] |
| 17:53 |
<moritzm> |
installing ncurses updates from stretch point release |
[production] |
| 17:19 |
<awight@tin> |
Started scap: Beta: Fix ORES thresholds and enable JADE, T181159, T176333 |
[production] |
| 17:06 |
<godog> |
cleanup integration-slave-jessie-1001:/srv/pbuilder/build - T189587 |
[production] |
| 16:45 |
<marostegui> |
Clean iptables rules on dbproxy1001 to leave it as dbproxy1006 |
[production] |
| 16:33 |
<marostegui> |
Retroactive: cleared iptables rules on dbproxy1007 |
[production] |
| 16:32 |
<jynus> |
restarting gerring on cobalt, stalled |
[production] |
| 16:26 |
<jynus> |
restarting gerring on cobalt, stalled |
[production] |
| 16:18 |
<jynus> |
update CNAME for m1-master and m2-master |
[production] |
| 15:50 |
<marostegui> |
Deploy schema change on db1097:3314 - T187089 T185128 T153182 |
[production] |
| 15:46 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Depool db1097:3314 for alter table (duration: 00m 56s) |
[production] |
| 15:39 |
<jynus> |
upgrade and restart dbproxy1007 |
[production] |
| 15:33 |
<vgutierrez> |
upgrading eqiad LVSs to pybal 1.15.2 |
[production] |
| 15:32 |
<jynus> |
upgrade and restart dbproxy1001 |
[production] |
| 14:55 |
<vgutierrez> |
upgrading codfw LVSs to pybal 1.15.2 |
[production] |
| 14:53 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Repool db1081 after alter table (duration: 00m 57s) |
[production] |
| 14:51 |
<jynus> |
stopping db2044 (this will make proxies complain about redundancy) |
[production] |
| 14:42 |
<moritzm> |
rebooting chromium for kernel security update |
[production] |
| 14:11 |
<chasemp> |
add chico to wmf-nda (verified nda things with moritz and all the goodness) |
[production] |
| 13:29 |
<jynus> |
stop db1001 for maintenance (proxies will temporarely complain about lack of redundancy) |
[production] |
| 13:20 |
<zeljkof> |
EU SWAT finished |
[production] |
| 13:20 |
<zfilipin@tin> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:417331|wmf-config: enable Singapore oversample as default on all wikis (T188652)]] (duration: 00m 57s) |
[production] |