2351-2400 of 10000 results (10ms)
2010-12-30 §
20:36 <Ryan_Lane> powercycling mobile2 - it's dead [production]
19:44 <apergos> add xml data fs to nfs in puppet, include that in snapshot hosts stanzas [production]
19:43 <apergos> remove andrew from snapshot stanzas in puppet, he should already be convered by home-no-service [production]
19:33 <awjr> pushed r79287 to payments cluster, replacing bad payflowpro_gateway.i18n.php with the good [production]
19:05 <awjr> pushed r79282 of DonationInterface to payments cluster, updating credit card validation on cc form, picking up i18n changes and new css for premiums [production]
18:49 <catrope> synchronized docroot/mediawiki/xml/export-0.4.xsd 'Update XSD file for XML dumps' [production]
15:59 <RobH> all blogs updated successfully, all plugins updated and reactivated [production]
15:48 <RobH> created backups of both blogs, pushing updates to new versions for security reasons [production]
11:27 <catrope> synchronized php-1.5/wmf-config/CommonSettings.php 'Bump style version appendix' [production]
11:25 <catrope> synchronized php-1.5/skins/common/shared.css 'r79244' [production]
06:40 <Tim> fixed my VOIP account [production]
2010-12-29 §
21:01 <richcole> drive15 from IBM db42 [production]
19:22 <Ryan_Lane> granted sudo privileges for demon on formey to create/modify/delete svn (LDAP) users [production]
02:03 <tomaszf> updating payments cluster to r79148 [production]
2010-12-28 §
21:54 <Ryan_Lane> destroying 1TB logical volume on storage3 [production]
21:54 <Ryan_Lane> umounting 1TB logical volume from /archive on storage3 and mounting 4TB xfs volume (data was moved prior to this) [production]
19:09 <Ryan_Lane> moving archive data from /archive to /archive1 on storage3, will replace mount afterwards [production]
19:08 <Ryan_Lane> creating new 4TB logical volume "archive1" on storage3, and putting an xfs filesystem on it [production]
16:34 <mark> Defined special disk space check (with SMS notification) for mysql core databases [production]
15:52 <midom> synchronized php-1.5/wmf-config/db.php 'adding db11, db18 and db39' [production]
14:30 <midom> synchronized php-1.5/wmf-config/db.php 'enabling db37' [production]
13:16 <domas> resumed replication on db11, db18, db37 and db39 [production]
00:19 <Ryan_Lane> restarting apache on srv217 - puppet updated its apache2.conf file, but didn't restart it [production]
00:12 <Ryan_Lane> powercycling amssq53 - it's dead [production]
00:09 <Ryan_Lane> powercycling sq76 - it's dead [production]
00:08 <Ryan_Lane> knsq14 isn't coming back up - may be broken [production]
00:08 <Ryan_Lane> powercycling knsq24 - it's dead [production]
00:02 <Ryan_Lane> powercycling knsq14 - it's dead [production]
2010-12-27 §
23:54 <Ryan_Lane> starting apache on srv217, running puppet to ensure code is synched [production]
23:52 <Ryan_Lane> restarting apache on srv231 [production]
23:50 <laner> synchronized php-1.5/wmf-config/mc.php 'Fixing IP address for srv226' [production]
23:48 <laner> synchronized php-1.5/wmf-config/mc.php 'Fixing IP address for srv227' [production]
23:39 <Ryan_Lane> powercycling srv217 - it's dead [production]
20:38 <Ryan_Lane> adding puppet schema to opendj on nova-controller.tesla [production]
19:51 <Ryan_Lane> specifically chgrp'd all files to svnadm and added write permissions for the group [production]
19:51 <Ryan_Lane> opened up file permissions on /srv/org/wikimedia/svn on formey so that ^demon can edit files [production]
19:48 <Ryan_Lane> disabled selenium service on windows7-1, and launched selenium services manually, while logged in as the selenium user [production]
18:04 <Ryan_Lane> installing puppet, puppet-el, puppetmaster, puppetmaster-passenger, and vim-puppet on nova-controller.tesla [production]
17:56 <Ryan_Lane> adding new 1TB "archive" logical volume on storage3, formatting as ext3, and mounting at /archive [production]
17:49 <Ryan_Lane> purging some bin logs on db9 to free up space [production]
16:27 <Ryan_Lane> installing graphviz on formey via puppet for bug 26404 [production]
09:21 <apergos> truncated log-all and log-index in /a/search/log on searchidx1 to get some space back [production]
2010-12-26 §
18:57 <Ryan_Lane> powercycling mobile1, as it is dead [production]
02:29 <Ryan_Lane> running /home/wikipedia/common/wmf-deployment/maintenance/purgeStaleMemcachedFlaggedRevs.php on s3/s7 flaggedrevs enabled wikis [production]
00:08 <ariel> synchronized wmf-deployment/wmf-config/db.php 'edits enabled for s3/s7, people should purge if they see weird things, flagged revs may still cause problems, hewiki still has issues' [production]
2010-12-25 §
23:39 <apergos> ran Platonides' script "purgeStaleMemcachedText.php" on s3/s7 projects, see /home/wikipedia/common/wmf-deployment/maintenance/purgeStaleMemcachedText.php on fenari (not added to the local branch yet) [production]
21:14 <ariel> synchronized wmf-deployment/wmf-config/db.php 'read-only for s3/s7 while we sort out edit sync and cache issues' [production]
20:17 <mark> New master bin-log db17-bin.003, position 406 [production]
20:16 <mark> Switched master on s3, s7 from db27 to db17, just one slave db25 [production]
20:16 <mark> synchronized php-1.5/wmf-config/db.php 'Going read-write on s3, s7' [production]