22:10 Kickfire Kickfire Kickfire (2862 Bytes) » MySQL Performance Blog

So it looks like these days Kickfire welcomed everyone to tell they’ve been working with Kickfire - Baron Keith Frank is writing about KickFire. Good Job Kickfire PR team you really energized community.
We also were working with Kickfire for quite a while and are also very excited to give it a try.

Enough have been written about technical basics by other people so I would not waste my time retelling.

I also prefer us to write about results of real testing rather than speculating on PR communications. Finding what products are good for, what are their sweet spots and limits is our mission. I will be playing hands on with one of these boxes next week but I guess I will not be able to publish any findings until after users conference.

Entry posted by peter | No comment

Add to: | | | |

20:36 临死想到追逐日光 (4019 Bytes) » Fenng's shared items in Google Reader

《追逐日光》
【美】尤金·奥凯利，中信出版社2007，26元。

去年公司和大学生打过一段交道，发现相当大一部分应届毕业生的求职目标是所谓“四大”。上海的学生有多么仰慕四大：财大的说，至少有三分之二的本科同学把进四大当成第一目标；复旦的说，他们金融系100多人，总有80多个想去四大吧；交大和上外的说，如果我们不想做本专业了，最好去四大或者宝洁这样不限专业的公司……

跟我们这代人当年的“四大天王”没有关系，“四大”是四家世界上最大的审计事务所——普华永道（PwC）、毕马威（KPMG）、德勤华永（DTT）、安永华明（EY）。

很难理解这么多本科生都喜欢做枯燥的审计工作。不过这么大范围的存在，应该有其合理之处，我们就做了一点了解。其吸引人之处，无外乎这样几条：
1、著名外企，进去有面子；
2、要人多，不限专业，进去可能性大（每家每年，只是在上海就要100～300的应届本科生——因为离职率高么）；
3、可以预期升职时间（前三年，很稳定的每年升一级。——简单说，就是把升职这件事做到了流水线化）；
4、收入可以较快达到中上水平（第一年5000，第二年7000，第三年10000。另外有出差补助。和其他外企相比，第一年算是普通，第二年第三年就胜出了。——反正不少人在第一年第二年就走了）
5、据说发展前景好，有人能跳到投行等让人留口水的行业（——人多了，有进什么行业的都不稀奇。总体看离职人员的去向，不见得比其他公司的出路要好）。

上面这些是“优点”。同学们对四大的缺点却也是如数家珍：加班的强度到了令人发指的程度，比很多私企有过之而无不及。再论到工作的重复和枯燥（第一年的主要工作就是复印、抽凭证、校对数字），则是放之四海，无人能敌。——但是这些阻挡不了他们追求梦想的脚步，前仆后继地去磕四大的门。

他们该读一读这本《追逐日光》。

毕马威美国前首席执行官，尤金·奥凯利，一直觉着脑袋不舒服，百忙万忙之中好容易抽了个时间，去了趟医院，医生说：脑癌晚期，无药可治，你还能活3个月，最多半年。这年是2005年，他53岁，被选上CEO还不到三年。

他开始筹划这三个月该怎么过：首先是辞职，然后数一数还有哪些没实现的梦想来得及实现，然后和朋友家人道别。同时还要写这本书。

他说，对自己这辈子还算满意，最后当了CEO，能和美国最牛的一些人一起吃晚饭。只是后悔花了太少时间陪家人，以及做自己想做的事——平均

Technorati 标签: 四大；书评；追逐日光

每周花在工作上的时间超过70个小时。

想进四大的同学们啊，毕马威在全球有近十万名员工，十万人都在加班，连老总都累死在岗位上了。真想不通你们进了四大有啥奔头！

18:25 使用_px_trace跟踪分析并行执行的情况 (2339 Bytes) » DBA@Taobao

Author:江枫 posted on Taobao.com

并行在系统资源充足的情况下，可以极大的加快操作的速度，在数据仓库环境中应用较多。而在OLTP环境中，由于并发较大，开启并行可能瞬间导致资源耗尽，所以一般只有在业务低估期间执行一些诸如创建索引等维护操作时才会考虑开启并行，并且在执行完成后去掉对象的并行度，否则可能后果很严重。

由于并行涉及到多个进程间分配协调任务，往往比较容易出现各种各样的问题，而且从数据字典中比较难以定位到具体的原因。Oracle提供了一些event来trace并行过程，如10384,10390,10399等等，但是这些event往往无法trace整个的并行过程，有时候需要设置多个event才能trace到我们需要的内容。而_px_trace则提供了一个统一的trace入口，并且有些信息还是event无法trace到的。

语法如下：

alter session set "_px_trace"=[[Verbosity,]area],[[Verbosity,]area],..,[time];

Verbosity表示trace信息的详细程度，取值为high,medium,low
area表示trace的区域，取值scheduling,execution,granul,messaging,buffer,compilation,all,none
time表示是否在trace中包含时间信息

SQL> alter session set "_px_trace"="compilation","execution","messaging","time";

会话已更改。

SQL> select count(*) from test;

COUNT(*)
----------
11846

SQL> alter session set "_px_trace"="none";

会话已更改。

生成的trace文件比较多，qc和slave进程都会生成相应的trace文件。具体的trace信息分析我这里就不写了，有兴趣的可以参考Metalink(Note:444164.1)

Add Comments(0)

18:21 使用_px_trace跟踪分析并行执行的情况 (3028 Bytes) » NinGoo@Net

Author:NinGoo posted on NinGoo.net

并行在系统资源充足的情况下，可以极大的加快操作的速度，在数据仓库环境中应用较多。而在OLTP环境中，由于并发较大，开启并行可能瞬间导致资源耗尽，所以一般只有在业务低估期间执行一些诸如创建索引等维护操作时才会考虑开启并行，并且在执行完成后去掉对象的并行度，否则可能后果很严重。

由于并行涉及到多个进程间分配协调任务，往往比较容易出现各种各样的问题，而且从数据字典中比较难以定位到具体的原因。Oracle提供了一些event来trace并行过程，如10384,10390,10399等等，但是这些event往往无法trace整个的并行过程，有时候需要设置多个event才能trace到我们需要的内容。而_px_trace则提供了一个统一的trace入口，并且有些信息还是event无法trace到的。

语法如下：

alter session set "_px_trace"=[[Verbosity,]area],[[Verbosity,]area],..,[time];

Verbosity表示trace信息的详细程度，取值为high,medium,low
area表示trace的区域，取值scheduling,execution,granul,messaging,buffer,compilation,all,none
time表示是否在trace中包含时间信息

SQL> alter session set "_px_trace"="compilation","execution","messaging","time";

会话已更改。

SQL> select count(*) from test;

COUNT(*)
----------
11846

SQL> alter session set "_px_trace"="none";

会话已更改。

生成的trace文件比较多，qc和slave进程都会生成相应的trace文件。具体的trace信息分析我这里就不写了，有兴趣的可以参考Metalink(Note:444164.1)

Related Articles

Add Comments(0)

18:08 Multi-Column IN clause - Unexpected MySQL Issue (15786 Bytes) » MySQL Performance Blog

We have an application which stores massive amount of urls. To save on indexes instead of using URL we index CRC32 of the URL which allows to find matching urls quickly. There is a bit of chance there would be some false positives but these are filtered out after reading the data so it works all pretty well.

If we just process urls one by one it works great:

PLAIN TEXT

SQL:

mysql> EXPLAIN SELECT url FROM 124pages.124pages WHERE url_crc=484036220 AND url="http://www.dell.com/";
+----+-------------+----------+------+---------------+---------+---------+-------+------+-------------+
| id | select_type | TABLE | type | possible_keys | KEY | key_len | ref | rows | Extra |
+----+-------------+----------+------+---------------+---------+---------+-------+------+-------------+
| 1 | SIMPLE | 124pages | ref | url_crc | url_crc | 4 | const | 1 | USING WHERE |
+----+-------------+----------+------+---------------+---------+---------+-------+------+-------------+
1 row IN SET (0.02 sec)

Handling URLs one by one is however not efficient if you're processing millions of them so we tried to do bulk fetches:

PLAIN TEXT

SQL:

mysql> EXPLAIN SELECT url FROM 106pages.106pages WHERE (url_crc, url) IN ((2752937066, 'http://members.aye.net/~gharris/blog/'), (3799762538, 'http://www.coxandforkum.com/'));
+----+-------------+----------+------+---------------+------+---------+------+-------+-------------+
| id | select_type | TABLE | type | possible_keys | KEY | key_len | ref | rows | Extra |
+----+-------------+----------+------+---------------+------+---------+------+-------+-------------+
| 1 | SIMPLE | 106pages | ALL | NULL | NULL | NULL | NULL | 14936 | USING WHERE |
+----+-------------+----------+------+---------------+------+---------+------+-------+-------------+
1 row IN SET (0.00 sec)

As you can see just using multiple column IN makes MySQL to pick doing full table scan in this case, even though the cardinality on the first column is almost perfect. I did some more testing and it looks like a bug or missing optimizer feature.

I should not be surprised though as multi-column in is not the most used MySQL feature out there.

For given application case we could simply rewrite query using more standard single column IN clause:

PLAIN TEXT

SQL:

mysql> EXPLAIN SELECT url FROM 106pages.106pages WHERE url_crc IN (2752937066,3799762538) AND url IN('http://members.aye.net/~gharris/blog/','http://www.coxandforkum.com/'); +----+-------------+----------+-------+---------------+---------+---------+------+------+-------------+
| id | select_type | TABLE | type | possible_keys | KEY | key_len | ref | rows | Extra |
+----+-------------+----------+-------+---------------+---------+---------+------+------+-------------+
| 1 | SIMPLE | 106pages | range | url_crc | url_crc | 4 | NULL | 2 | USING WHERE |
+----+-------------+----------+-------+---------------+---------+---------+------+------+-------------+
1 row IN SET (0.01 sec)

Theoretically speaking this query is not equivalent to the first one - because row having url_crc=2752937066 and url='http://www.coxandforkum.com/' would match it, while it should not. It however does not happen in our case as url_crc is functionally dependent on url so both queries are equivalent.

So we've got our work around and can forget about the issue and MySQL team gets yet another bug to deal with.
What worries me again is - this is very simple case which seems to to be generally broken which raises a question how good coverage MySQL tests have.

Entry posted by peter | One comment

Add to: | | | |

16:31 Oracle确定连接方式的优先级 (625 Bytes) » yangtingkun

前两天在使用10g的简化连接方式的时候突然想到的问题，如果TNSNAMES.ORA中的配置名称恰好是另一个数据库的简化连接方式，Oracle会连接到哪个数据库。今天有时间，做了个简单的测试。首先通过简化连接方式连接远端的数据库：SQL> CONN YANGTK/YANGTK@172.25.1.100/TEST1已连接。SQL> SELECT * FROM GLOBAL_NAME;GLOBAL_NAME-----------------------------------------------------TEST1.YANGTINGKUN下面在本地的TNSNAMES.ORA中配置另一个数据库，但是配置的名称为172.25.1.100/TEST1：172.25.1.100/test1 = (DESCRIPTION = (ADDRESS = (PRO...

12:04 Friday round-up (2304 Bytes) » Red Hat Magazine

An interesting development in intellectual property law, the Superman comics copyright case, is in the news again. The heirs of the original creators of Superman were given rights to Action Comics #1, a decision that splits ownership of the Superman universe. Who owns large-scale works created over time? How are they divided? In this case, some characters are owned by one entity and others by another. What are the implications for “written works” like software and applications? Interesting questions to ponder…
Is there such a thing as technological terrorism? You might scoff, since it’s true that most dire warnings about the dangers of the internet seem silly. But the recent Wired piece about an epilepsy discussion board hit with griefers posting flashing graphics is a bit alarming.
Of interest: a piece from Linux Journal on the popularity of blogs as promotional media. With a catchy title: Blogging vs flogging.
The PaperWorld 3D source is now open. PaperWorld lets you create 3D massively multiplayer online gaming (MMOG) environments that run in Flash.

11:30 Improved Snapshot interface for MySQL backups (233 Bytes) » O'Reilly Databases

We have significantly improved the snapshot interface for doing MySQL backups using ZRM. This work has been released as part of ZRM 2.0. ZRM 2.0 has couple of snapshot plugins - Linux LVM and Solaris ZFS that uses the interface....

10:23 Oracle11g ASMCMD新命令 (14950 Bytes) » NinGoo@Net

Author:NinGoo posted on NinGoo.net

Oracle10g的ASMCMD命令，提供了通过命令行方式管理ASM的接口，但是功能非常有限，比如无法在asm和os之间直接复制文件，就是一件很让人头痛的事情，只能通过rman或者dbms_file_transfer实现。

Oracle11g的ASMCMD终于加上了一个比较实用的cp命令，不但可以在ASM和OS之间复制文件，也可以在不同的ASM Instance和Diskgroup之间复制文件，这就非常的方便了。

ASMCMD> cp +dgtest/test/datafile/USERS.264.646186565 users.dbf
source +dgtest/test/datafile/USERS.264.646186565
target users.dbf
copying file(s)...
file, E:\ORACLE\PRODUCT\11.1.0\DB_1\DATABASE\USERS.DBF, copy committed.

但是有点头痛的是，在windows平台上，不知道如何指定OS的路径，因为cp命令语法中，冒号被用作连接字符串和具体路径之间的分隔符了，有知道怎么解决的朋友请留言告知，谢谢。

ASMCMD> help cp
cp [-ifr] <[\@connect_identifier:]src> <[\@connect_identifier:]tgt>

Oracle10g对于ASM的metadata的保护是非常弱的，基本上没有提供常规的手段来备份或者恢复metadata，这样一旦出现metadata损坏之类的问题，就会非常的麻烦。11g的ASMCMD新增加了md_backup/md_restore两个命令，对ASM的metadata提供了备份恢复的功能。

ASMCMD> md_backup -b meta.bak -g dgtest
Disk group to be backed up: DGTEST

从备份出来的文件看，ASM的metadata是相当的简单的：

@diskgroup_set = (
{
   'ATTRINFO' => {
   'AU_SIZE' => '1048576',
   'DISK_REPAIR_TIME' => '3.6h',
   'COMPATIBLE.ASM' => '11.1.0.0.0',
   'COMPATIBLE.RDBMS' => '10.1.0.0.0'
   },
   'DISKSINFO' => {
   'DGTEST_0001' => {
   'DGTEST_0001' => {
   'TOTAL_MB' => '1024',
   'FAILGROUP' => 'DGTEST_0001',
   'NAME' => 'DGTEST_0001',
   'DGNAME' => 'DGTEST',
   'PATH' => 'E:\\ASM\\DISK2'
   }
   },
   'DGTEST_0000' => {
   'DGTEST_0000' => {
   'TOTAL_MB' => '1024',
   'FAILGROUP' => 'DGTEST_0000',
   'NAME' => 'DGTEST_0000',
   'DGNAME' => 'DGTEST',
   'PATH' => 'E:\\ASM\\DISK1'
   }
   }
   },
   'DGINFO' => {
   'DGTORESTORE' => 0,
   'DGCOMPAT' => '11.1.0.0.0',
   'DGNAME' => 'DGTEST',
   'DGDBCOMPAT' => '10.1.0.0.0',
   'DGTYPE' => 'EXTERN',
   'DGAUSZ' => '1048576'
   },
   'ALIASINFO' => {},
   'TEMPLATEINFO' => {
   '6' => {
   'DGNAME' => 'DGTEST',
   'STRIPE' => 'COARSE',
   'TEMPNAME' => 'ASM_STALE',
   'REDUNDANCY' => 'UNPROT',
   'SYSTEM' => 'Y'
   },
   '11' => {
   'DGNAME' => 'DGTEST',
   'STRIPE' => 'FINE',
   'TEMPNAME' => 'FLASHBACK',
   'REDUNDANCY' => 'UNPROT',
   'SYSTEM' => 'Y'
   },
   '3' => {
   'DGNAME' => 'DGTEST',
   'STRIPE' => 'COARSE',
   'TEMPNAME' => 'ARCHIVELOG',
   'REDUNDANCY' => 'UNPROT',
   'SYSTEM' => 'Y'
   },
   '7' => {
   'DGNAME' => 'DGTEST',
   'STRIPE' => 'COARSE',
   'TEMPNAME' => 'BACKUPSET',
   'REDUNDANCY' => 'UNPROT',
   'SYSTEM' => 'Y'
   },
   '9' => {
   'DGNAME' => 'DGTEST',
   'STRIPE' => 'COARSE',
   'TEMPNAME' => 'XTRANSPORT',
   'REDUNDANCY' => 'UNPROT',
   'SYSTEM' => 'Y'
   },
   '2' => {
   'DGNAME' => 'DGTEST',
   'STRIPE' => 'FINE',
   'TEMPNAME' => 'CONTROLFILE',
   'REDUNDANCY' => 'UNPROT',
   'SYSTEM' => 'Y'
   },
   '12' => {
   'DGNAME' => 'DGTEST',
   'STRIPE' => 'COARSE',
   'TEMPNAME' => 'DATAGUARDCONFIG',
   'REDUNDANCY' => 'UNPROT',
   'SYSTEM' => 'Y'
   },
   '8' => {
   'DGNAME' => 'DGTEST',
   'STRIPE' => 'COARSE',
   'TEMPNAME' => 'AUTOBACKUP',
   'REDUNDANCY' => 'UNPROT',
   'SYSTEM' => 'Y'
   },
   '4' => {
   'DGNAME' => 'DGTEST',
   'STRIPE' => 'FINE',
   'TEMPNAME' => 'ONLINELOG',
   'REDUNDANCY' => 'UNPROT',
   'SYSTEM' => 'Y'
   },
   '1' => {
   'DGNAME' => 'DGTEST',
   'STRIPE' => 'COARSE',
   'TEMPNAME' => 'DUMPSET',
   'REDUNDANCY' => 'UNPROT',
   'SYSTEM' => 'Y'
   },
   '0' => {
   'DGNAME' => 'DGTEST',
   'STRIPE' => 'COARSE',
   'TEMPNAME' => 'PARAMETERFILE',
   'REDUNDANCY' => 'UNPROT',
   'SYSTEM' => 'Y'
   },
   '10' => {
   'DGNAME' => 'DGTEST',
   'STRIPE' => 'COARSE',
   'TEMPNAME' => 'CHANGETRACKING',
   'REDUNDANCY' => 'UNPROT',
   'SYSTEM' => 'Y'
   },
   '13' => {
   'DGNAME' => 'DGTEST',
   'STRIPE' => 'COARSE',
   'TEMPNAME' => 'TEMPFILE',
   'REDUNDANCY' => 'UNPROT',
   'SYSTEM' => 'Y'
   },
   '5' => {
   'DGNAME' => 'DGTEST',
   'STRIPE' => 'COARSE',
   'TEMPNAME' => 'DATAFILE',
   'REDUNDANCY' => 'UNPROT',
   'SYSTEM' => 'Y'
   }
   }
}
;

Related Articles

Add Comments(0)

09:42 Log Buffer #91: a Carnival of the Vanities for DBAs (2661 Bytes) » Pythian Group Blog » Log Buffer

Welcome to the 91^st edition of Log Buffer, the weekly review of database blogs.

For a change, let’s begin with some PostgreSQL stuff. On Tending the Garden, Selena Deckelmann gives her retrospective thanks to those who attended and presented the PostgreSQL Conference East.

On Esoteric Curio, Theo Schlossnagle gives his thoughts on the keynote address by Joshua Drake, touching on the perennial versus, Postgres vs. MySQL.

Hey, there was a MySQL ambassador there, too — Baron Schwartz of xaprb. Here’s Baron’s recap of his experiences at the conference.

When pet projects bite back! reasserts that SQL is in fact a programming language. Sometimes one can forget that and need a little reminder. Or a not-so-little reminder, such as a three-pages-long query. The discussion ranges into questions of design, a matter that Baron Schwartz also pursues: he asks (on behalf of his wife), what is your favorite database design book? (I want to know too — um, for . . . a friend of mine.) Lots of good responses so far.

For huge queries to huge tables. On the MySQL performance blog, Aurimas Mikalauskas walks us through using MMM to ALTER huge tables. He writes, “When it comes to changes that really require table to be rebuilt - adding/dropping columns or indexes, changing data type, converting data to different character set - MySQL master-master replication especially accompanied by MMM can be very handy to do the changes with virtually no downtime.”

(more…)

07:18 How do you use Fedora? (1684 Bytes) » Red Hat Magazine

Paul Frields, Fedora Project Leader, asks:

Fedora is being used in a hundred thousand interesting places out in the real world.

* At NASA to do analysis that makes future space exploration safer.
* In studies to improve development for children with Down’s Syndrome (trisomy 21).
* At Creative Commons, to promote open content discovery, creation, and remixing.
* Powering distribution of food to the needy in Mexico.

We want to hear a story about how YOU use Fedora to do something interesting. Enriching. Educational. Enhancing. At work or in your community. How did Fedora help you achieve something special? Just reply in a comment to [his] blog.