0%

source: https://www.swyx.io/learn-in-public/

  • a habit of creating learning exhaust
    • Write blogs and tutorials and cheatsheets.
    • Speak at meetups and conferences.
    • Ask and answer things on Stackoverflow or Reddit.
      • Avoid the walled gardens like Slack and Discord, they’re not public.
    • Make Youtube videos or Twitch streams.
    • Start a newsletter.
    • Draw cartoons (people loooove cartoons!)
  • Whatever your thing is, make the thing you wish you had found when you were learning.
  • Don’t judge your results by “claps” or retweets or stars or upvotes
  • Oh you think you’re done? Don’t stop there:
    • Enjoyed a coding video? Reach out to the speaker/instructor and thank them, and ask questions.
    • Make PR’s to libraries you use.
    • Make your own libraries no one will ever use.
    • Clone stuff you like, from scratch, to see how they work.
    • Teach workshops.
    • Go to conferences and summarize what you learned.
  • The subheading under this rule would be: Try your best to be right, but don’t worry when you’re wrong.
    • People think you suck? Good. You agree. Ask them to explain, in detail, why you suck
    • You want to just feel good or you want to be good?
      • Then go away and prove them wrong. Of course, if they get abusive block them.
  • At some point you’ll get some support behind you. People notice genuine learners. They’ll want to help you.
    • Don’t tell them, but they just became your mentors.
    • This is very important: Pick up what they put down
    • Because you learn in public. By teaching you, they teach many.

使用 Logseq 写笔记已经大半年了,Logseq 是这么多年来除了 vim 之外,唯一可以在使用上螺旋上升的软件。我把 Logseq 推荐给媳妇,她用来写会议纪要,写备忘录,管理待办事项,也用得挺好。

Logseq 的成功之处在于它降低了记录的成本,不需要大段大段的文字,也不需要太多的文章结构,随时都可以记录。Logseq 比较严重的问题是它的使用太灵活了,前后记录的标准、格式、标签等要素可能不一致,容易造成笔记凌乱。笔记一凌乱就没法有效的聚合,笔记的有效聚合形成一个有意义的想法群是 Logseq 等双链软件和其他笔记软件相比最有价值的地方。

我这半年来总结出笔记系统成功的关键是: 在同一个地方,用同样的格式和一致的标准记录你的洞见。但是由于 Logseq 缺乏有效的工作流,没法自动化提示必要的步骤,只能通过不断的练习来强化学习。另外 Logseq 中 page 和 tag 只是起到过滤笔记的作用,最关键的笔记之间的链接却很容易被忽视,tag 的数量一多又会开始笔记凌乱了。

总而言之,如果只是把 Logseq 作为备忘,或者简单作为信息记录,挺好用也不需要学习太多的东西,但是如果需要使用 Logseq 来实践卡片笔记法等 PKM 理论,恐怕还是需要花费一些功夫仔细琢磨一番。PKM 方面的书籍我强力推荐 《卡片笔记写作法》一书,中文翻译很不错,虽然书名是谈写作,里面的内容却涉及很广,我自己阅读后收益良多。

就实践卡片笔记法来说,浮墨是一个比较理想的工具,「少即是多,多则惑」,去掉一切不需要的功能,重点才能突出。Logseq 的野心挺大,不想只做一个双链笔记本,还想做得更多,这就要求我们想清楚使用其的主要目的是什么。现实中并不存在一个 all in one 的理想工具,什么工具合适就可以使用什么工具,各位施主不能 「着相」了。

上面说的是一些「道」层面的东西, Logseq 在「术」方面的东西也不少,下面将介绍一些前两篇尚未涉及的功能和使用技巧。

☆ 使用 properties

https://docs.logseq.com/#/page/term%2Fproperties

在 Logseq 中有两种 properties, page properties 和 block properties。

  • Page properties 在第一个 block 写 property:: value
  • Block properties 在每个 block 内使用 property:: value

在使用 block properties 时,需要输入 shift + Enter 来进行换行,换行后输入 :: Logseq 会自动提示已经使用过的 property。理解 properties 可以简单的把 block 看成数据库的一条记录, properties 就是不同的字段,可以通过 Logseq 的 query 来查询相关记录。在使用 properties 时可以引用 page 或者 tag,可以通过这种方法来索引这条记录。

例如,我将书籍的 block 都添加了 category:: Books 的属性,后续可以使用 {{query (property category Books) }} 来查询所有书籍了,感觉上有点类似 notion 的表格。

☆ 使用 /Scheduled 制定计划

使用 Schedule 的功能的方法比较简单,在写完一条 block 后,输入 shift + enter 换行后,输入 /Scheduled 在弹出的界面中选择时间周期即可。

在 Schedule 规定的时间内,Logseq 的 Journals 页面可以看到相关的内容。

☆ 使用模板 (Template)

https://docs.logseq.com/#/page/templates

使用模板的好处是可以避免重复性的劳动,比如你希望在 Journals 页面里按照 [[每日工作]] [[每日心得]] 的固定格式来记录,那么你需要在每天重复输入一次。但如果你制作一个固定模板,就可以使用 /Template 命令来完成自动化的输入。

制作模板的过程可以参考 Logseq 官方的动画,简单说就是写一个 block,这个 block 包含了必要的结构,在 block 前面的圆点点击鼠标右键 -> Make template,再输入模板的名字就完成了。在后续的使用过程中,输入 /Template 后在弹出的界面选择对应的模板名就可以完成自动化的输入。

☆ 一些好用的插件

插件可以拓展 Logseq 的功能,使用插件的方法在 Logseq 使用小结 (二) 中有详细介绍,这里就不再细说。

Logseq Plugin Tabs

https://github.com/pengx17/logseq-plugin-tabs

Plugin Tabs 像打开浏览器一样打开 pages 或者 blocks,对于内容比较多的 page,这个插件非常实用。

logseq-plugin-mark-map

https://github.com/vipzhicheng/logseq-plugin-mark-map

Mark map 可以自动化地把你的笔记用导图的形式输出,方便展示和记忆。

☆ 一些使用上的小技巧

  • Logseq content wide mode 可以使用快捷键切换 t w
  • Logseq 的 property 可以通过输入 :: 来触发自动补全操作,非常方便。
  • Logseq 使用快捷键 Cmd + Shift + i 可以打开 devtools
  • Logseq v0.68,自动补全提供两个快捷键 mod+pmod+n 比较方便了
  • Logseq 可以使用 command + command - 来放大或者缩小 Logseq 的字体大小
  • Logseq 0.7.1 新功能:Copy & Paste with rich-text formats,如果粘贴时不想保留格式 mod + shift + v
  • Logseq 移动光标的快捷键:mod + p 向上移动光标,mod + n 向下移动光标
  • Logseq Markdown 语法支持 Horizontal Rules,可以使用 ---分隔卡片笔记。

mod 在 windows 和 linux 系统中是 ctrl, 在 macOS 系统中是 command

☆ 参考链接

https://www.usmacd.com/2022/02/21/logseq/
https://www.usmacd.com/2022/03/07/logseq2/

  • Random Notes (漫游笔记) 的技术原理是用新视角去审视旧想法,和自己以前的想法不期而遇的感觉非常奇妙。总体来说人是会不断进步的,经过一段时间后再去重温自己某个时刻的想法,可能以前的一些疑惑现在已经没有了,也可能对某个问题有了更进一步的思路,这是促使想法迭代的一种方法。

  • 想要做到想法的自然迭代,必要的条件是可以马上回到过去的某个时刻,而且那个时刻的上下文没有丢失,如果上下文丢失意味着已经看不懂以前的笔记了,自然也就没有了迭代的可能性。

  • 卡片盒笔记法要求用完整的句子精要简述想法,笔记之间的连接则补充了上下文,为想法迭代创造了有利条件。

  • 写永久笔记时,应该假设读者对文本背后的思想,原文背景一无所知,只具备相关的领域知识。这里的读者其实也包括未来的自己,我们写下永久笔记后,很快就会将其的上下文遗忘,就想从没有见过该笔记的其他读者一样。

  • 我们应该寻找与我们观点相反的论点或者事实来挑战我们的既有思维,但是受「确认偏差」的影响,我们会不经意的忽略这些信息,那些和我们观点一致的信息有很大吸引力,因为这些信息可以证明自己很博学。解决 「确认偏差」的方法:把寻找证实性信息变为收集所有信息,不需要关注这些信息支持的具体观点。

  • 我们使用卡片盒笔记法,对习惯最重要的转变是将注意力从个别项目的预设立场变成思考笔记之间的开放性联系。通过一段时间的训练,在阅读过程中我们可以轻松找出与预设观点相悖的信息。当这些信息改变我们对某些问题的看法时,我们会感到相当兴奋,从而慢慢喜欢上这种感觉。

  • 在实践卡片笔记法的过程中,会遇上一整天都没写一张卡片的时候,而有的时候则一天写十多张卡片。限制每天写笔记的数量可以让写卡片盒笔记的过程更加轻松,避免了过于放松的状态,也避免了过于紧绷的状态,有利于我们保持良好的状态。另一个好处则是让你选择笔记内容时更加挑剔,有利于提高笔记的质量。

  • 你可以将目标设置为每天写3条笔记,或者每年出一本书,仍然可以在合理的时间内积累大量的想法。

  • 卡片笔记法属于自下而上的发散思维,发散思维有个问题,就是无法把握发散的度。为了应对这个问题,卢曼提出的办法是仔细思考笔记之间的联系,写笔记时参考文献笔记和自己以前写的笔记,定期自上而下制作索引卡片,这些做法的目的是避免想法过于太发散。

  • 思维需要发散,但是领域需要聚焦,我们在自己的专业,财务等领域是需要不断精进的,这两个方面似乎是矛盾的。有人建议使用分类法,把关注的领域分类,在日常工作学习中不断完善,但这种方法很容易回到资料归档的老路。

  • 最近的研究发现可以使用 P.A.R.A. 来组织顶层结构,统领需要精进的领域,在具体想法的记录上则继续使用卡片笔记法。这种方法使用 P.A.R.A. 不断推动自己在关注领域的取得进展,又能在关键的思考上使用卡片笔记法,自用一段时间感觉不错。

  • 关于笔记管理存在几个误区,1. 笔记软件/博客系统,我先后使用过 EverNote,WizNote,VNote,CSDN blog,Google blogspot, WordPress 2. 笔记格式,我先后使用过 txt, orgmode, markdown 其实这些都不是关键,除了折腾还是折腾。以前我常说一个段子,你以为你在写博客,其实你在折腾 Wordpress 插件。

  • 我以前思考这个问题认为是笔记缺乏组织结构,实践结果表明良好的分类确实有一定帮助,但是不是最重要的。最重要的是在同一个地方,用同样的格式和一致的标准记录你的洞见。

  • 把笔记保存在同一个地方避免了笔记分散,迁移也比较方便。同样的格式和一致的标准则为笔记之间的联系与聚合创造了有利条件。如果笔记之间格式不一致,粒度不一致,不容易产生新的想法。另外,当我们以一致的标准写笔记时阻力是最小的,关注的是笔记的内容,形式等外在的东西则变得不再重要。新的观点、新的洞见、新的想法,是笔记系统中最有价值的东西。笔记系统应该侧重于知识的积累而不是信息的积累。

  • 各种笔记软件,博客系统,笔记格式的切换都会带来信息丢失。信息丢失容易造成心理负担,我从 2005 年开始写文章到现在已经丢失了 50%以上,这些文章散落在互联网的各个角落,非常令人痛心。

未完,待续 …

有关金融市场方面的书我个人的图书馆里至少有1200本。它们主要是讨论证券分析、期权策略、期货策略、技术分析以及其他相关方面的书。这些书大多都包含相当不错的构想,其中约2%是真正优秀的作品。然而,它们大多存在一个共同的问题:试图推销一种“战胜市场”的方法,很多方法甚至没有经过实际市场的完整测试。

如何正确的绘制趋势线

使用以下方法绘制趋势线可以提供一致而精确的结果,而且绝对不会产生错误的信号:

  1. 选择考虑的时间:长期(数月至数年)、中期(数个星期至数月)或短期(数天至数个星期)。
  2. 上升趋势线:在考虑的时间内,以最低的低点为起点,向右上方绘制一条直线,连接最高点前的某一个低点,使这条直线在两个低点之间未穿越任何价位。延伸这条直线而经过最高点(这是指水平轴上的位置而言)。趋势线经过所考虑的最高点以后,它可能穿越某些价位。事实上,这是趋势发生变化的一种现象。
  3. 下降趋势线:在考虑的时间内,以最高的高点为起点,向右下方绘制一条直线,连接最低点前的某一个高点,而使这条直线在两个高点之间未穿越任何价位。延伸这条直线而经过最低点。

这个方法的优点在于其明确性,并可以用来判断趋势是否可能变动或已经变动。

确立趋势变动的 123 法则

  • 股价突破趋势线
  • 上升趋势回档后,不创新高;下降趋势反弹后,不创新低
  • 上升趋势跌破上一个回档低点,下降趋势突破上一个反弹高点

风险报酬比很不错的 2B法则

上升趋势中价格创新高后,并未持续上涨,而后又跌破先前的高点,则趋势很可能反转。

具有统计学优势的四天准则

在中期走势中,当市场在高点或者低点以连续四天下跌或上涨的走势而呈反转时,趋势可能发生变化。 (和趋势方向相反的连续四根K线)

四天准则辅助准则

当中期走势发展到相当程度以后,如果出现顺势的四天(或者以上)排列,随后出现第一天的逆趋势行情,经常代表趋势变动的顶部或者底部。

三天高低价准则

三天的高低价准则,是运用最近三天的盘中高价与低价。当价格发生反转而穿越三天的高价或低价时,则做多或卖空,并以第三天的低价或高价作为止损点。 这个准则必须配合另一个确认原则。

缺口准则

缺口准则很简单,当趋势线上方或者下方出现缺口时,它反映重大变化(消息面或基本面),并显示趋势可能发生变动。该准则并不需要2B,也不需要经过试探(趋势线)。
然而,缺口必须穿越趋势线,该准则才有效。 这个准则必须配合另一个确认原则。

0. 下载 Visual Studio Community 2022

下载地址:https://aka.ms/vs/17/release/vs_community.exe

从微软网站下载并安装 Visual Studio Community 的时候速度只有2 K,一下午没能下载成功。咨询同事小钻风,指出可能是 DNS 问题,将 DNS 切换成阿里的 DNS 223.5.5.5 后下载速度达到 4 M。中国开发者网络总是个问题,大家都挺不容易的。

1. 编译 winafl

  1. 下载 DynamoRIO,https://github.com/DynamoRIO/dynamorio/releases/download/cronbuild-9.0.19209/DynamoRIO-Windows-9.0.19209.zip 解压到 D:\DynamoRIO
  2. 下载 cmake,https://github.com/Kitware/CMake/releases/download/v3.24.0/cmake-3.24.0-windows-x86_64.zip 解压到 D:\cmake
  3. 从 Github 下载 winafl 源码 https://github.com/googleprojectzero/winafl.git , 编译 intel PT 需要同步 third_party 的源码
1
2
3
git clone https://github.com/googleprojectzero/winafl.git
cd winafl
git submodule update --init --recursive
  1. 从菜单中选择 X64 native tool command prompt for VS 2022,执行编译命令
1
2
3
4
mkdir build64
cd build64
D:\cmake\bin\cmake.exe -G"Visual Studio 17 2022" -A x64 .. -DDynamoRIO_DIR=D:\DynamoRIO\cmake -DINTELPT=1
D:\cmake\cmake.exe --build . --config Release

-DINTelPT=1 参数将编译 Intel PT 模式的支持,编译完成后,可以正常运行

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
.\build64\bin\Release>.\afl-fuzz.exe

WinAFL 1.16b by <ifratric@google.com>
Based on AFL 2.43b by <lcamtuf@google.com>

.\afl-fuzz.exe [ afl options ] -- [instrumentation options] -- \path\to\fuzzed_app [ ... ]

Required parameters:

-i dir - input directory with test cases
-o dir - output directory for fuzzer findings
-t msec - timeout for each run

Instrumentation type:

-D dir - directory with DynamoRIO binaries (drrun, drconfig)
-w winafl - Path to winafl.dll
-P - use Intel PT tracing mode
-Y - enable the static instrumentation mode

Execution control settings:

-f file - location read by the fuzzed program (stdin)
-m limit - memory limit for the target process
-p - persist DynamoRIO cache across target process restarts
-c cpu - the CPU to run the fuzzed program

Fuzzing behavior settings:

-d - quick & dirty mode (skips deterministic steps)
-n - fuzz without instrumentation (dumb mode)
-x dir - optional fuzzer dictionary (see README)

Other stuff:

-I msec - timeout for process initialization and first run
-T text - text banner to show on the screen
-M \ -S id - distributed mode (see parallel_fuzzing.txt)
-C - crash exploration mode (the peruvian rabbit thing)
-e - expert mode to run WinAFL as a DynamoRIO tool
-l path - a path to user-defined DLL for custom test cases processing
-V - show version number and exit

Attach:

-A module - attach to the process that loaded the provided module

For additional tips, please consult afl_docs\README.


C:\Users\zhouzhen\Downloads\winafl\build64\bin\Release>

2. 检查 DynamoRIO 模式是否正常工作

使用 IDA Pro 对 test_gdiplus.exe 进行反汇编,target_offset 为 0x10E0

1
2
3
.text:00000001400010E0 ; int __cdecl main(int argc, const char **argv, const char **envp)
.text:00000001400010E0 main proc near ; CODE XREF: __scrt_common_main_seh(void)+107↓p
.text:00000001400010E0 ; DATA XREF: .rdata:0000000140002B6C↓o ...

先用 drrun.exe 测试 winafl 是否正常工作,其实就是循环执行 10 次目标程序。

1
2
3
D:\DynamoRIO\bin64\drrun.exe -c winafl.dll -debug ^
-target_module test_gdiplus.exe -target_offset 0x10E0 -fuzz_iterations 10 ^
-nargs 2 -- test_gdiplus.exe input.bmp

执行命令后,在当前目录下会生成一个 log 文件,文件名为 afl.test_gdiplus.exe.14172.0000.proc.log,内容如下:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
Module loaded, dynamorio.dll
Module loaded, winafl.dll
Module loaded, drx.dll
Module loaded, drreg.dll
Module loaded, drmgr.dll
Module loaded, drwrap.dll
Module loaded, test_gdiplus.exe
Module loaded, gdiplus.dll
Module loaded, VCRUNTIME140.dll
Module loaded, msvcp_win.dll
Module loaded, win32u.dll
Module loaded, ucrtbase.dll
Module loaded, KERNELBASE.dll
Module loaded, gdi32full.dll
Module loaded, KERNEL32.dll
Module loaded, msvcrt.dll
Module loaded, IMM32.dll
Module loaded, RPCRT4.dll
Module loaded, GDI32.dll
Module loaded, combase.dll
Module loaded, USER32.dll
Module loaded, ntdll.dll
In pre_fuzz_handler
Module loaded, UxTheme.dll
Module loaded, OLEAUT32.dll
Module loaded, bcrypt.dll
Module loaded, SECHOST.dll
Module loaded, ADVAPI32.dll
Module loaded, WindowsCodecs.dll
Module loaded, MSCTF.dll
In post_fuzz_handler
In pre_fuzz_handler
In post_fuzz_handler
In pre_fuzz_handler
In post_fuzz_handler
In pre_fuzz_handler
In post_fuzz_handler
In pre_fuzz_handler
In post_fuzz_handler
In pre_fuzz_handler
In post_fuzz_handler
In pre_fuzz_handler
In post_fuzz_handler
In pre_fuzz_handler
In post_fuzz_handler
In pre_fuzz_handler
In post_fuzz_handler
In pre_fuzz_handler
In post_fuzz_handler
Everything appears to be running normally.
Coverage map follows:

Everything appears to be running normally. 表明 winafl 自己认为没有出错。 log 文件中还包含了其他有用信息,比如说目标程序加载的模块。这些加载的模块可以用来设置 -coverage_module来统计你所关注的 coverage。

使用 drrun.exe 验证成功后,可以执行下面命令测试 winafl 是否正常工作。

1
2
3
cmd > afl-fuzz.exe -i in -o out -D D:\DynamoRIO\bin64 -t 20000 ^
-- -coverage_module gdiplus.dll -coverage_module WindowsCodecs.dll -fuzz_iterations 5000 ^
-target_module test_gdiplus.exe -target_offset 0x10E0 -nargs 2 -- test_gdiplus.exe @@

执行后上面的命令后,如果发现 total paths 的数字在不断变化, winafl 就已经可以正常工作了,恭喜 😄

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
+- process timing -------------------------------------+- overall results ----+
| run time : 0 days, 0 hrs, 0 min, 1 sec | cycles done : 0 |
| last new path : 0 days, 0 hrs, 0 min, 1 sec | total paths : 2 |
| last uniq crash : none seen yet | uniq crashes : 0 |
| last uniq hang : none seen yet | uniq hangs : 0 |
+- cycle progress --------------------+- map coverage -+----------------------+
| now processing : 0 (0.00%) | map density : 1.65% / 1.65% |
| paths timed out : 0 (0.00%) | count coverage : 1.00 bits/tuple |
+- stage progress --------------------+ findings in depth --------------------+
| now trying : arith 8\8 | favored paths : 1 (50.00%) |
| stage execs : 388/389 (99.74%) | new edges on : 2 (100.00%) |
| total execs : 619 | total crashes : 0 (0 unique) |
| exec speed : 353.3/sec | total tmouts : 0 (0 unique) |
+- fuzzing strategy yields -----------+---------------+- path geometry -------+
| bit flips : 0/56, 1/55, 0/53 | levels : 2 |
| byte flips : 0/7, 0/6, 0/4 | pending : 2 |
| arithmetics : 0/0, 0/0, 0/0 | pend fav : 1 |
| known ints : 0/0, 0/0, 0/0 | own finds : 1 |
| dictionary : 0/0, 0/0, 0/0 | imported : n/a |
| havoc : 0/0, 0/0 | stability : 99.63% |
| trim : 0.00%/1, 0.00% +-----------------------+
^C----------------------------------------------------+ [cpu000001: 14%]

3. DynamoRIO 模式 winafl.dll 各个参数的具体含义

winafl.dll 是 DynamoRIO client (instrumentation) code,参数挺多 Winafl 的 README 中介绍了下面这些参数。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
-covtype         - the type of coverage being recorded. Supported options are
bb (basic block, default) or edge.

-coverage_module - module for which to record coverage. Multiple module flags
are supported.

-target_module - module which contains the target function to be fuzzed.
Either -target_method or -target_offset need to be
specified together with this option.

-target_method - name of the method to fuzz in persistent mode. For this to
work either the method needs to be exported or the symbols
for target_module need to be available. Otherwise use
-target_offset instead.

-target_offset - offset of the method to fuzz from the start of the module.

-fuzz_iterations - Maximum number of iterations for the target function to run
before restarting the target process.

-nargs - Number of arguments the fuzzed method takes. This is used
to save/restore the arguments between runs.

-call_convention - The default calling convention is cdecl on 32-bit x86
platforms and Microsoft x64 for Visual Studio 64-bit
applications. Possible values:
* fastcall: fastcall
* ms64: Microsoft x64 (Visual Studio)
* stdcall: cdecl or stdcall
* thiscall: thiscall

-debug - Debug mode. Does not try to connect to the server. Outputs
a log file containing loaded modules, opened files and
coverage information.

-logdir - specifies in which directory the log file will be written
(only to be used with -debug).

-thread_coverage - If set, WinAFL will only collect coverage from a thread
that executed the target function
  • -nargs target method 或者 target offset 的参数个数
  • -fuzz_iterations 重启进程前,循环执行 target method 或者 target offset 的次数
  • -target_moudle fuzz 的目标模块,需要和 target method 或者 target offset 一起使用
  • -target_method persistent mode 下 fuzz 的函数名(方法名),需要有符号 (symbols)
  • -target_offset fuzz 的函数(方法)相对于模块起始地址的偏移
  • -coverage_module 统计覆盖率的模块,可以统计多个模块的 coverage,即指定多个 -coverage_module 参数
  • -covtype bb 或者 edge,bb (basic block 为默认值)
  • -thread_coverage 用于多线程程序,只统计一个线程执行 target function 的覆盖率变化

4. WinAFL intel PT 模式使用

Intel PT (Processor Tracing) 是 Intel CPU 的一个特性,从 Intel 第5代处理器开始支持,5th generation 的微架构为 Broadwell。相关说明可以参考:https://www.intel.com/content/www/us/en/support/articles/000056730/processors.html

Windows 10 从 v1809 开始提供了 Intel PT 的驱动,但目前没有公开的文档描述此驱动,也没有提供官方的 API。 Alex Ionescu 提供了 winipt library 和 Intel PT 驱动交互,WinAFL 利用 winipt 来获得 Intel PT 的 Processo Tracing 信息。

WinAFL 使用 Intel PT 模式,和 DynamoRIO 不同需要指定 -P , WinAFL Intel PT 模式除了和DynamoRIO 相同的参数外额外添加了下面几个参数:

  • -trace_size 每次执行 traget 手机的 trace 信息的字节数,必须是2 的次方并且大于 4096
  • -decode 处理 trace 信息的解码器,有三个可选值:tip, tip_reffull (full 为默认值)
  • -nopersistent_trace 由于性能的缘故,WinAFL 每次循环执行 target 是不重启程序,这个选项可以强制 WinAFL 每次执行 target 都重启,一般是为了调试才会启用这个选项
  • -trace_cache_size trace 信息的 cache 字节数,和 full 解码器一起使用

5 检查 Intel PT 模式是否正常工作

1
2
winaflpt-debug.exe -debug -coverage_module test_gdiplus.exe -fuzz_iterations 10 ^
-target_module test_gdiplus.exe -target_offset 0x10E0 -nargs 2 -- test_gdiplus.exe @@

上面命令其实和检查 DynamoRIO 模式是否正常工作的命令基本一直,执行命令后输出如下:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
Module loaded: test_gdiplus.exe
Module loaded: ntdll.dll
Module loaded: KERNEL32.DLjL
Module loaded: KERNELBASE.dll
Module loaded: ucrtbase.dll
Module loaded: gdiplus.dll
Module loaded: msvcrt.dll
Module loaded: combase.dll
Module loaded: VCRUNTIME140.dll
Module loaded: RPCRT4.dll
Module loaded: USER32.dll
Module loaded: win32u.dll
Module loaded: GDI32.dll
Module loaded: gdi32full.dll
Module loaded: msvcp_win.dll
Module loaded: IMM32.DLL
iteration 0
Module loaded: uxtheme.dll
Module loaded: msctf.dll
Module loaded: oleaut32.dll
Module loaded: sechost.dll
Iteration finished normally
iteration 1
Iteration finished normally
iteration 2
Iteration finished normally
iteration 3
Iteration finished normally
iteration 4
Iteration finished normally
iteration 5
Iteration finished normally
iteration 6
Iteration finished normally
iteration 7
Iteration finished normally
iteration 8
Iteration finished normally
iteration 9
Iteration finished normally
Coverage map (hex):
00000000000000000000000000000000
00000000000000000000000000000000

6. 用 Intel PT 模式 fuzz

1
afl-fuzz.exe -i in -o out -P -t 20000 -- -coverage_module test_gdiplus.exe -fuzz_iterations 10 -target_module test_gdiplus.exe -target_offset 0x10E0 -nargs 2 -- test_gdiplus.exe @@

用上面的命令调用 afl-fuzz,发现可以正常工作,界面如下:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

WinAFL 1.16b based on AFL 2.43b (test_gdiplus.exe)

+- process timing -------------------------------------+- overall results ----+
| run time : 0 days, 0 hrs, 0 min, 13 sec | cycles done : 10 |
| last new path : none yet (odd, check syntax!) | total paths : 1 |
| last uniq crash : none seen yet | uniq crashes : 0 |
| last uniq hang : none seen yet | uniq hangs : 0 |
+- cycle progress --------------------+- map coverage -+----------------------+
| now processing : 0 (0.00%) | map density : 0.04% / 0.04% |
| paths timed out : 0 (0.00%) | count coverage : 1.07 bits/tuple |
+- stage progress --------------------+ findings in depth --------------------+
| now trying : havoc | favored paths : 1 (100.00%) |
| stage execs : 48/128 (37.50%) | new edges on : 1 (100.00%) |
| total execs : 2207 | total crashes : 0 (0 unique) |
| exec speed : 160.8/sec | total tmouts : 0 (0 unique) |
+- fuzzing strategy yields -----------+---------------+- path geometry -------+
| bit flips : 0/32, 0/31, 0/29 | levels : 1 |
| byte flips : 0/4, 0/3, 0/1 | pending : 0 |
| arithmetics : 0/222, 0/2, 0/0 | pend fav : 0 |
| known ints : 0/19, 0/102, 0/40 | own finds : 0 |
| dictionary : 0/0, 0/0, 0/0 | imported : n/a |
| havoc : 0/1664, 0/0 | stability : 100.00% |
| trim : 42.86%/1, 0.00% +-----------------------+
^C----------------------------------------------------+ [cpu000001: 18%]

参考资料

https://github.com/googleprojectzero/winafl/blob/master/README.md
https://github.com/googleprojectzero/winafl/blob/master/readme_dr.md
https://github.com/googleprojectzero/winafl/blob/master/readme_pt.md

source: https://null2root.github.io/blog/2022/07/21/When-Hypervisor-Met-Snapshot-Fuzzing.html

1. Introduction

Hypervisor was known as hard target to fuzz over several years. Even though, lots of prior pioneers( Peter Hlavaty, Chaitin Tech, StarLabs, Peleg Hadar and Ophir Harpaz and many others ) doing amazing work to overcome this limit and found interesting bugs.

But it is not an easy topic for some fresh newbie like me to starts from the bottom. I think manual code( or binary ) auditing and static analysis could be only considerable options if I start my research a few years ago without What The Fuzz.

What The Fuzz( a.k.a WTF ) is a snapshot-based fuzzer targeting Windows Ring-3 and Ring-0 component. WTF’s snapshot is based on memory dump both kernel and user-land. Because of that, WTF can not emulate any functionality which requires anything not within in-memory and can not create new process or thread because memory dump limited on single process. But WTF support breakpoint Handler which you can set breakpoint on any address within memory dump and if fuzz execution reaches that address, pre-defined breakpoint handler will be executed. Based on this, we can trying to overcome some limitation on WTF such as file access.

I really love WTF’s flexibility and its potential, and I am going to show one of example usage of WTF on targeting Virtualbox to prove how awesome it is.

2. Developing Fuzz Module

First, you should define your own fuzz module for your target. There are few examples on github( fuzzer_hevd.cc, fuzzer_tlv_server.cc ) and blog post( ret2systems, Doar-e ).

My Target was Virtualbox’s SVGA component.

SVGA is something like “JavaScript of hypervisors“. Because of its complex nature, many hypervisor bugs are oriented by SVGA. And that’s the reason why I choose it as a first target.

VirtualBox have a function called vmsvgaR3FifoLoop. It waits until guest submit new command data through GPA. So this is a good spot( or should I call it source? ) to take a snapshot.

I set a breakpoint on vmsvgaR3FifoLoop+4E2 to take snapshot. It is same position as here, start of switch-case routine for SVGA and SVGA3D command.

blahblah1

After creating snapshot, I had to decide make it run multiple times in single execution or not. Because SVGA is consist of various commands like define, update, delete something…, I thought fuzz campaign must handle multiple SVGA commands during single round.

First, I had to define structure to contains multiple testcases. Luckily, 0vercl0k already write nice example to doing that. So I did same thing as below.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
#define SVGA_CMD_BODY_MAX 0xA000

const uint32_t SvgaCmdList[] = {1, 3, 19, 22, 25, 29, 30, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 1040, 1041, 1042, 1043, 1044, 1046, 1047, 1048, 1049, 1050, 1051, 1052, 1053, 1054, 1055, 1056, 1057, 1058, 1059, 1060, 1061, 1062, 1063, 1064, 1065, 1066, 1067, 1068, 1069, 1070, 1071, 1072, 1073, 1074, 1075, 1076, 1077, 1078, 1079, 1080, 1081};

const uint32_t SvgaCmdListLen = 58;

struct SvgaCmd_t {
uint32_t SvgaCmdCode;
std::vector<uint8_t> SvgaBody;
NLOHMANN_DEFINE_TYPE_INTRUSIVE(SvgaCmd_t, SvgaCmdCode, SvgaBody);
};

struct SvgaCmds_t {
std::vector<SvgaCmd_t> SvgaCmds;
NLOHMANN_DEFINE_TYPE_INTRUSIVE(SvgaCmds_t, SvgaCmds);
};

struct {
std::deque<SvgaCmd_t> SvgaCmds;
CpuState_t Context;
uint8_t SvgaCmdBody[SVGA_CMD_BODY_MAX];
uint32_t SvgaCmdBodySize;

void RestoreGprs(Backend_t *B) {
const auto &C = Context;
B->Rsp(C.Rsp);
B->Rip(C.Rip);
B->Rax(C.Rax);
B->Rbx(C.Rbx);
B->Rcx(C.Rcx);
B->Rdx(C.Rdx);
B->Rsi(C.Rsi);
B->Rdi(C.Rdi);
B->R8(C.R8);
B->R9(C.R9);
B->R10(C.R10);
B->R11(C.R11);
B->R12(C.R12);
B->R13(C.R13);
B->R14(C.R14);
B->R15(C.R15);
}
} GlobalState;

SvgaCmds_t Deserialize(const uint8_t *Buffer, const size_t BufferSize) {
const auto &Root = json::json::parse(Buffer, Buffer + BufferSize);
return Root.get<SvgaCmds_t>();
}

bool InsertTestcase(const uint8_t *Buffer, const size_t BufferSize) {
GlobalState.SvgaCmds.clear();

const auto &Root = Deserialize(Buffer, BufferSize);
for(auto SvgaCmd : Root.SvgaCmds) {
GlobalState.SvgaCmds.emplace_back(std::move(SvgaCmd));
}

return true;
}

It is almost identical as example.

Next question was, how can I insert data into snapshot? I had to find a insertion point and proper memory address. Luckily( again ), VirtualBox using a function called vmsvgaR3FifoGetCmdPayload to receive command data from guest to host. I define a breakpoint handler in Init() callback function as below.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
//
// unsigned __int8 *__fastcall vmsvgaR3FifoGetCmdPayload(
// unsigned int cbPayloadReq,
// volatile unsigned int *pFIFO,
// unsigned int offCurrentCmd,
// unsigned int offFifoMin,
// unsigned int offFifoMax,
// unsigned __int8 *pbBounceBuf,
// unsigned int *pcbAlreadyRead,
// PDMTHREAD *pThread,
// VGAState *pThis,
// VMSVGAR3STATE *pSVGAState,
// PDMDEVINSR3 *pDevIns)
//
if(!g_Backend->Setbreakpoint("VBoxDD!vmsvgaR3FifoGetCmdPayload", [](Backend_t *Backend) {
uint32_t cbPayloadReq = Backend->GetArg(0);
Gva_t pbBounceBuf_gva = Backend->GetArgGva(5);
Gva_t pcbAlreadyRead_gva = Backend->GetArgGva(6);

if(cbPayloadReq > GlobalState.SvgaCmdBodySize) {
DebugPrint("cbpayloadReq({:#x}) > SvgaCmdBodySize({:#x}), restore context and goto next round\n", cbPayloadReq, GlobalState.SvgaCmdBodySize);
return GlobalState.RestoreGprs(Backend);
}

if(cbPayloadReq > u32PbBounceBufMaxSize) {
const uint64_t RetAddr = Backend->VirtRead8(Gva_t(Backend->Rsp()));
DebugPrint("check this, RetAddr = {:#x}\n", RetAddr);
std::abort();
}

if(!Backend->VirtWriteDirty(pbBounceBuf_gva, GlobalState.SvgaCmdBody, cbPayloadReq)) {
fmt::print("Failed to write pbBounceBuf, pbBounceBuf = {:#x}, cbPayloadReq = {:#x}, SvgaCmdBodySize = {:#x}\n",
pbBounceBuf_gva.U64(), cbPayloadReq, GlobalState.SvgaCmdBodySize);
std::abort();
}

if(!Backend->VirtWriteStructDirty(pcbAlreadyRead_gva, &cbPayloadReq)) {
fmt::print("Faile to write pcbAlreadyRead, pcbAlreadyRead = {:#x}\n", pcbAlreadyRead_gva.U64());
std::abort();
}

Backend->SimulateReturnFromFunction(pbBounceBuf_gva.U64());

})) {
fmt::print("Failed to Setbreakpoint on VBoxDD!vmsvgaR3FifoGetCmdPayload\n");
return false;
}

I also had to define end point of execution. After some reversing, I found two spots and using it as end point.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
const Gva_t SvgaLoopEnd = Gva_t(g_Dbg.GetSymbol("VBoxDD!vmsvgaR3FifoLoop") + 0x24AB);
if(!g_Backend->Setbreakpoint(SvgaLoopEnd, [](Backend_t *Backend) {
DebugPrint("loop end reached, restore context\n");
GlobalState.RestoreGprs(g_Backend);
})) {
fmt::print("Failed to Setbreakpoint on SvgaLoopEnd\n");
return false;
}

const Gva_t SvgLoopEnd2 = Gva_t(g_Dbg.GetSymbol("VBoxDD!vmsvgaR3FifoLoop") + 0x24B2);
if(!g_Backend->Setbreakpoint(SvgLoopEnd2, [](Backend_t *Backend) {
DebugPrint("loop end2 reached, restore context\n");
GlobalState.RestoreGprs(g_Backend);
})) {
fmt::print("Failed to Setbreakpoint on SvgLoopEnd2\n");
return false;
}

As I said above, I create a snapshot on vmsvgaR3FifoLoop+4E2. So if I restore register context, next execution flow starts from there. Because of that, I had to parse new testcase using breakpoint Handler.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
const Gva_t SvgaLoopStart = Gva_t(g_Dbg.GetSymbol("VBoxDD!vmsvgaR3FifoLoop") + 0x4e2);
if(!g_Backend->Setbreakpoint(SvgaLoopStart, [](Backend_t *Backend) {
if(GlobalState.SvgaCmds.size() == 0) {
DebugPrint("testcase deque empty! goto next round...\n");
return Backend->Stop(Ok_t());
}

auto &Testcase = GlobalState.SvgaCmds.front();

while (Testcase.SvgaCmdCode == 1045) {
GlobalState.SvgaCmds.pop_front();

if(GlobalState.SvgaCmds.size() == 0) {
DebugPrint("testcase deque empty during cmd filtering! goto next round...\n");
return Backend->Stop(Ok_t());
}

Testcase = GlobalState.SvgaCmds.front();
}

Backend->Rbx(Testcase.SvgaCmdCode);

DebugPrint("SvgaCmdCode = {:#x}\n", Testcase.SvgaCmdCode);

if(Testcase.SvgaCmdCode >= 1040) {

DebugPrint("Have to avoid AssertBreak(pHdr->size < pThis->svga.cbFIFO), write {:#x} on first DWORD\n", Testcase.SvgaBody.size());

const uint32_t Svga3dCmdSize = Testcase.SvgaBody.size();
if(Svga3dCmdSize >= 0x200000) {
fmt::print("Svga3dCmdSize({:#x}) > cbFIFO, abort\n", Svga3dCmdSize);
std::abort();
}

memcpy(&GlobalState.SvgaCmdBody[0], &Svga3dCmdSize, 4);
memcpy(&GlobalState.SvgaCmdBody[4], Testcase.SvgaBody.data(), Testcase.SvgaBody.size());
GlobalState.SvgaCmdBodySize = Testcase.SvgaBody.size() + 4;
}
else {
memcpy(GlobalState.SvgaCmdBody, Testcase.SvgaBody.data(), Testcase.SvgaBody.size());
GlobalState.SvgaCmdBodySize = Testcase.SvgaBody.size();
}

GlobalState.SvgaCmds.pop_front();
})) {
fmt::print("Failed to Setbreakpoint on SvgaLoopStart\n");
return false;
}

After some investigation, I found CreateDeviceEx call in vmsvga3dContextDefine keep causing CR3 context switching and I didn’t found any way to handling it using breakpoint handler. So I just blacklisting it( SVGA_3D_CMD_CONTEXT_DEFINE = 1045 ).

I disclose almost every part of fuzz module except mutation part. I just use libfuzzer mutator to mutate body data of SVGA command and pick one of command in array randomly.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
explicit CustomMutator_t(std::mt19937_64 &Rng, const size_t TestcaseMaxSize)
: Rng_(Rng), TestcaseMaxSize_(TestcaseMaxSize) {

// set maximum size for multiple (mutated) testcase( Default = 1MB )
ScratchBuffer__ = std::make_unique<uint8_t[]>(_1MB);
ScratchBuffer_ = {ScratchBuffer__.get(), _1MB};

BodyMutator_ = make_unique<LibfuzzerMutator_t>(Rng, TestcaseMaxSize);
}

// skip for brevity...

std::string Mutate(uint8_t *Data, const size_t DataLen, const size_t MaxSize) {
auto Root = Deserialize(Data, DataLen);
auto &SvgaCmds = Root.SvgaCmds;

for(auto &SvgaCmd : SvgaCmds) {

//
// 50%
//
if(GetUint32(0, 1) == 1) {
uint32_t SvgaCmdRandomIdx = GetUint32(0, SvgaCmdListLen);
SvgaCmd.SvgaCmdCode = SvgaCmdList[SvgaCmdRandomIdx];
}

memcpy(GlobalMutBuffer, SvgaCmd.SvgaBody.data(), SvgaCmd.SvgaBody.size());
size_t NewTestcaseSize = BodyMutator_->Mut_.Mutate(GlobalMutBuffer, SvgaCmd.SvgaBody.size(), MaxSize);
SvgaCmd.SvgaBody.resize(NewTestcaseSize);
memcpy(SvgaCmd.SvgaBody.data(), GlobalMutBuffer, NewTestcaseSize);
}

json::json Serialized;
to_json(Serialized, Root);
return Serialized.dump();
}

I also define generator function. It is very useful if you are too lazy to create random input like me ^~^.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
std::string GenerateTestcase() {
SvgaCmds_t Root;
const auto N = GetUint32(1, 10);

for (size_t Idx = 0; Idx < N; Idx++) {
SvgaCmd_t SvgaCmd;

SvgaCmd.SvgaCmdCode = SvgaCmdList[GetUint32(0, SvgaCmdListLen)];
SvgaCmd.SvgaBody.resize(GetUint32(0x100, 0x400));

if(GetUint32(0, 1) == 0) {
for(int i = 0; i < SvgaCmd.SvgaBody.size(); i++) {
SvgaCmd.SvgaBody[i] = 0x41;
}
}
else {
for(int i = 0; i < SvgaCmd.SvgaBody.size(); i++) {
SvgaCmd.SvgaBody[i] = (uint8_t)GetUint32(0, 255);
}
}

Root.SvgaCmds.emplace_back(SvgaCmd);
}

json::json Serialized;
to_json(Serialized, Root);
return Serialized.dump();
}

std::string GetNewTestcase(const Corpus_t &Corpus) override {

//
// 20%
//
if(GetUint32(1, 5) == 1) {
return GenerateTestcase();
}

const Testcase_t *Testcase = Corpus.PickTestcase();

if (!Testcase) {
fmt::print("The corpus is empty, generate random one\n");
return GenerateTestcase();
}

//
// Copy the input in a buffer we're going to mutate.
//

memcpy(ScratchBuffer_.data(), Testcase->Buffer_.get(),
Testcase->BufferSize_);

// return Mutate(ScratchBuffer_.data(), Testcase->BufferSize_, TestcaseMaxSize_);

return Mutate(ScratchBuffer_.data(), Testcase->BufferSize_, u32PbBounceBufMaxSize - sizeof(uint32_t));
}

Aaaaannnd, this is everything you need to fuzz! I skip some formal code for brevity, but I think you can easily find what you need to define fully working fuzz module.

…..Ooooh wait, I forgot something. After some hours of struggle, I define a blacklist which cause CR3 context switching. I just put it on the end of Init() function.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
bool SetupVBoxBlacklistHooks() {
if(!g_Backend->SetBreakpoint("VBoxRT!RTLogLoggerEx", [](Backend_t *Backend) {
Backend->SimulateReturnFromFunction(0);
})) {
fmt::print("Failed to SetBreakpoint on VBoxRT!RTLogLoggerEx\n");
std::abort();
}

if(!g_Backend->SetBreakpoint("VBoxVMM!PDMCritSectLeave", [](Backend_t *Backend) {
Backend->SimulateReturnFromFunction(0);
})) {
fmt::print("Failed to SetBreakpoint on VBoxVMM!PDMCritSectLeave\n");
std::abort();
}

if(!g_Backend->SetBreakpoint("VBoxC!util::AutoLockBase::release", [](Backend_t *Backend) {
Backend->SimulateReturnFromFunction(0);
})) {
fmt::print("Failed to SetBreakpoint on VBoxC!util::AutoLockBase::release\n");
std::abort();
}

if(!g_Backend->SetBreakpoint("VBoxC!util::AutoLockBase::acquire", [](Backend_t *Backend) {
Backend->SimulateReturnFromFunction(0);
})) {
fmt::print("Failed to SetBreakpoint on VBoxC!util::AutoLockBase::acquire\n");
std::abort();
}

if(!g_Backend->SetBreakpoint("VirtualBoxVM!UIFrameBufferPrivate::NotifyChange", [](Backend_t *Backend) {
Backend->SimulateReturnFromFunction(0);
})) {
fmt::print("Failed to SetBreakpoint on VirtualBoxVM!UIFrameBufferPrivate::NotifyChange\n");
std::abort();
}

if(!g_Backend->SetBreakpoint("VBoxRT!SUPSemEventWaitNoResume", [](Backend_t *Backend) {
Backend->SimulateReturnFromFunction(0);
})) {
fmt::print("Failed to SetBreakpoint on VBoxRT!SUPSemEventWaitNoResume\n");
std::abort();
}

if(!g_Backend->SetBreakpoint("d3d9!CBaseDevice::Release", [](Backend_t *Backend) {
Backend->SimulateReturnFromFunction(0);
})) {
fmt::print("Failed to SetBreakpoint on d3d9!CBaseDevice::Release\n");
}

if(!g_Backend->SetBreakpoint("d3d9!CD3DBase::Clear", [](Backend_t *Backend) {
Backend->SimulateReturnFromFunction(0);
})) {
fmt::print("Failed to SetBreakpoint on d3d9!CD3DBase::Clear\n");
}

if(!g_Backend->SetBreakpoint("d3d9!CMipMap::SetAutoGenFilterType", [](Backend_t *Backend) {
Backend->SimulateReturnFromFunction(0);
})) {
fmt::print("Failed to SetBreakpoint on d3d9!CMipMap::SetAutoGenFilterType\n");
}

if(!g_Backend->SetBreakpoint("d3d9!CMipMap::GenerateMipSubLevels", [](Backend_t *Backend) {
Backend->SimulateReturnFromFunction(0);
})) {
fmt::print("Failed to SetBreakpoint on d3d9!CMipMap::GenerateMipSubLevels\n");
}

if(!g_Backend->SetBreakpoint("USER32!GetDC", [](Backend_t *Backend) {
Backend->SimulateReturnFromFunction(0x1337);
})) {
fmt::print("Failed to SetBreakpoint on USER32!GetDC\n");
}

return true;
}

Yep. That’s really everything. I use this fuzz module several hours and it founds interesting crash.

3. Vulnerability( TL;DR )

Crash occurred in vmsvga3dSurfaceCopy function( PageHeap needed ).

blahblah2

This function trying to copy surface data from one to another using surface id and there’s no boundary check between surface, so it become exploitable wildcopy vulnerability in heap memory.

spot the bug

This vulnerability patched in 6.1.36 release at July, 2022.

4. Conclusion

I think importance of snapshot fuzzing is, it makes researcher to focus on target itself.

Unlike other fuzzers based on runtime and DBI are often create (very) unreasonable side effect or need lots of time to create working harness. The concept of snapshot fuzzing makes it possible to reduce this waste of time.

可以量化一下,有时间考虑程序实现。

  • 涨/跌停板数
  • 涨停封版成功率
  • 涨跌股票比率
  • 大盘成交量额
  • 融资余额情况
  • 北向资金净流入 (参考)

  • 定投只有在低估的时候买,当估值进入中位数以上就不要买了,同时定投标的的估值应该在历史底部区域。
  • 定投的本质不是价值投资,而是相信估值回归中位数。
  • 指数基金,周期性行业适合定投,比如券商,钢铁,煤炭。一般按时间定投,如果大幅下跌也要买。

总的来说定投属于策略型的投资方法,其实并不知道底和顶在哪里,通过拉长时间来降低持仓成本,因此定投成功的关键在于你的成本是否可以随着时间而降下来。从这点也可以分析出在底部区域定投比在顶部区域定投要好得多,这也就是第一条。

定投策略对历史的估值情况有很强的依赖,属于基于历史数据的统计分析,但是每一次的情况都不同,没有人可以保证历史一定会重演。定投也不是价值投资,没有护城河,只是相信均值回归,这是第二条。

巴菲特非常推荐标普500指数 (S&P 500) 定投,指数基金最牛的地方是股票会定期调整,把好的股票加入进来,差的淘汰出去,相当于有人帮你选股了,另外足够分散,发生系统性风险的概率大幅降低。周期性行业则是爆发性很强,三年不开张,开张吃三年,容易在底部积累大量筹码,作为一个长期投资策略非常合适。

Fedora 36 正式版已经释出一段时间了,根据我的经验开始的一周都会有 bug,一般等一段时间再升级会更平滑一些。然而,老革命又遇上了新问题,显示出问题了启动不起来。我已经很久没有遇上显示驱动问题了,我在很久以前就禁用了开源驱动 nouveau,一直都用得很好。

修改 /etc/default/grub 文件,在 GRUB_CMDLINE_LINUX 添加下面参数

1
GRUB_CMDLINE_LINUX="... rd.driver.blacklist=nouveau modprobe.blacklist=nouveau nvidia-drm.modeset=1"

重新生成 grub 文件, sudo grub2-mkconfig -o /boot/grub2/grub.cfg,但是在升级 Fedora 36 后似乎没有起作用。诡异的是 Fedora 35 的内核却可以正常工作,首先怀疑的是 nvidia 驱动可能没有安装对。执行下面命令行

1
2
sudo dnf reinstall xorg-x11-drv-nvidia akmod-nvidia kmod-nvidia.x86_64
sudo reboot

可惜没有解决问题,上网找资料看看 Fedora 36 有没什么特殊的地方。安装 Nvidia 驱动网上有这篇文章说的详细
https://www.linuxcapable.com/how-to-install-nvidia-drivers-on-fedora-36-linux/
仔细拜读了一下,还是没有发现啥问题,驱动也重新安装了啊,开始怀疑是新版 Nvidia 驱动的问题

sudo dnf search xorg-x11-drv-nvidia 可以查看所有的驱动,xorg-x11-drv-nvidia-340xx xorg-x11-drv-nvidia-340xx xorg-x11-drv-nvidia-340xx 不知道到底需要安装哪个版本的驱动。😭

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
xorg-x11-drv-nvidia.x86_64 : NVIDIA's proprietary display driver for NVIDIA graphic cards
=========================================================================== 名称 和 概况 匹配:xorg-x11-drv-nvidia
xorg-x11-drv-nvidia-340xx-cuda.x86_64 : CUDA libraries for xorg-x11-drv-nvidia-340xx
xorg-x11-drv-nvidia-340xx-devel.i686 : Development files for xorg-x11-drv-nvidia-340xx
xorg-x11-drv-nvidia-340xx-devel.x86_64 : Development files for xorg-x11-drv-nvidia-340xx
xorg-x11-drv-nvidia-340xx-kmodsrc.x86_64 : xorg-x11-drv-nvidia-340xx kernel module source code
xorg-x11-drv-nvidia-340xx-libs.i686 : Libraries for xorg-x11-drv-nvidia-340xx
xorg-x11-drv-nvidia-340xx-libs.x86_64 : Libraries for xorg-x11-drv-nvidia-340xx
xorg-x11-drv-nvidia-390xx-cuda.x86_64 : CUDA driver for xorg-x11-drv-nvidia-390xx
xorg-x11-drv-nvidia-390xx-cuda-libs.i686 : CUDA libraries for xorg-x11-drv-nvidia-390xx
xorg-x11-drv-nvidia-390xx-cuda-libs.x86_64 : CUDA libraries for xorg-x11-drv-nvidia-390xx
xorg-x11-drv-nvidia-390xx-devel.i686 : Development files for xorg-x11-drv-nvidia-390xx
xorg-x11-drv-nvidia-390xx-devel.x86_64 : Development files for xorg-x11-drv-nvidia-390xx
xorg-x11-drv-nvidia-390xx-kmodsrc.x86_64 : xorg-x11-drv-nvidia-390xx kernel module source code
xorg-x11-drv-nvidia-390xx-libs.i686 : Libraries for xorg-x11-drv-nvidia-390xx
xorg-x11-drv-nvidia-390xx-libs.x86_64 : Libraries for xorg-x11-drv-nvidia-390xx
xorg-x11-drv-nvidia-470xx-cuda.x86_64 : CUDA driver for xorg-x11-drv-nvidia-470xx
xorg-x11-drv-nvidia-470xx-cuda-libs.i686 : CUDA libraries for xorg-x11-drv-nvidia-470xx
xorg-x11-drv-nvidia-470xx-cuda-libs.x86_64 : CUDA libraries for xorg-x11-drv-nvidia-470xx
xorg-x11-drv-nvidia-470xx-devel.i686 : Development files for xorg-x11-drv-nvidia-470xx
xorg-x11-drv-nvidia-470xx-devel.x86_64 : Development files for xorg-x11-drv-nvidia-470xx
xorg-x11-drv-nvidia-470xx-kmodsrc.x86_64 : xorg-x11-drv-nvidia-470xx kernel module source code
xorg-x11-drv-nvidia-470xx-libs.i686 : Libraries for xorg-x11-drv-nvidia-470xx
xorg-x11-drv-nvidia-470xx-libs.x86_64 : Libraries for xorg-x11-drv-nvidia-470xx

可是仔细一琢磨, Fedora 35 Nvidia 驱动不是工作正常吗,可以上去看看用的是哪个版本的驱动,执行命令 nvidia-settings,在 System Information 中发现使用的就是最新版本的驱动 510.68.02,看样子最新版本的驱动也没有问题,一下陷入了困境。😰

没啥好办法,还是 Google 搜索一下,结果发现了一个链接在讨论这个问题,和我遇上的现象一样,作者最终解决了问题
Fedora Linux kernel update breaks NVIDIA driver

原来是旧版的驱动没有卸载干净,参考 https://rpmfusion.org/Howto/NVIDIA#Uninstall_the_NVIDIA_driver
驱动卸载 Nvidia 驱动需要使用命令 dnf remove xorg-x11-drv-nvidia\*

按照下面的命令操作了一遍,重启系统,熟悉的登录界面终于又出现了。😄

1
2
3
4
sudo dnf remove xorg-x11-drv-nouveau
sudo dnf remove xorg-x11-drv-nvidia\*
sudo dnf install xorg-x11-drv-nvidia akmod-nvidia kmod-nvidia.x86_64
sudo reboot

What are protocol buffers?

Protocol buffers are Google’s language-neutral, platform-neutral, extensible mechanism for serializing structured data – think XML, but smaller, faster, and simpler. You define how you want your data to be structured once, then you can use special generated source code to easily write and read your structured data to and from a variety of data streams and using a variety of languages.

这段是 Google 官方网站给出的介绍,protobuf 可以自动化生成代码,用于读入或者写入结构化数据。一个简单的 protobuf 文件可以是这样的:

1
2
3
4
5
message Person {
required string name = 1;
required int32 id = 2;
optional string email = 3;
}

具体的语法可以参考Google 的文档 proto2proto3。c++ 语言使用 protobuf 的示例可以参见Protocol Buffer Basics: C++ 文档,基本步骤总结如下:

  1. 写 protobuf 文件,表达数据结构
  2. 利用 protoc 自动生成代码 (支持多种语言 C++ Java 等)
  3. 利用生成的文件解析或者写入相关数据结构

libprotobuf-mutator

编译安装

参考: https://github.com/google/libprotobuf-mutator/blob/master/README.md

1
2
3
4
5
git clone https://github.com/google/libprotobuf-mutator.git
mkdir build
cd build
cmake .. -GNinja -DCMAKE_C_COMPILER=clang -DLIB_PROTO_MUTATOR_DOWNLOAD_PROTOBUF=ON -DCMAKE_CXX_COMPILER=clang++ -DCMAKE_BUILD_TYPE=Debug
https_proxy=http://127.0.0.1:3128 ninja check

由于需要从Google下载一些源码,所以在 ninja check 的时候需要挂上代理,结果编译出错了,找不到 libxml2.a,排查一下编译参数,发现需要添加编译静态库的参数

1
2
3
4
5
6
7
8
9
10
11
12
diff --git a/cmake/external/libxml2.cmake b/cmake/external/libxml2.cmake
index c00ace2..a944fab 100644
--- a/cmake/external/libxml2.cmake
+++ b/cmake/external/libxml2.cmake
@@ -38,6 +38,7 @@ ExternalProject_Add(${LIBXML2_TARGET}
UPDATE_COMMAND ""
CONFIGURE_COMMAND ${LIBXML2_SRC_DIR}/autogen.sh --without-python
--prefix=${LIBXML2_INSTALL_DIR}
+ --enable-static=yes
CC=${CMAKE_C_COMPILER}
CXX=${CMAKE_CXX_COMPILER}
CFLAGS=${LIBXML2_CFLAGS}

修改后可以正常通过 ninja check 命令的所有检查。默认情况下 ninja install 会安装到 /usr/local 目录,因为考虑到后续需要给 afl+使用,所以需要使用下的命令重新 cmake 一下

1
2
cmake .. -GNinja -DCMAKE_C_COMPILER=clang -DLIB_PROTO_MUTATOR_DOWNLOAD_PROTOBUF=ON \
-DCMAKE_INSTALL_PREFIX=/home/henices/code/AFL+/external/libprotobuf-mutator -DCMAKE_CXX_COMPILER=clang++ -DCMAKE_BUILD_TYPE=Debug

libprotobuf-mutator 代码实现

libfuzzer_macro.h

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
#define DEFINE_TEST_ONE_PROTO_INPUT_IMPL(use_binary, Proto)                 \
extern "C" int LLVMFuzzerTestOneInput(const uint8_t* data, size_t size) { \
using protobuf_mutator::libfuzzer::LoadProtoInput; \
Proto input; \
if (LoadProtoInput(use_binary, data, size, &input)) \
TestOneProtoInput(input); \
return 0; \
}

// Defines custom mutator, crossover and test functions using default
// serialization format. Default is text.
#define DEFINE_PROTO_FUZZER(arg) DEFINE_TEXT_PROTO_FUZZER(arg)
// Defines custom mutator, crossover and test functions using text
// serialization. This format is more convenient to read.
#define DEFINE_TEXT_PROTO_FUZZER(arg) DEFINE_PROTO_FUZZER_IMPL(false, arg)
// Defines custom mutator, crossover and test functions using binary
// serialization. This makes mutations faster. However often test function is
// significantly slower than mutator, so fuzzing rate may stay unchanged.
#define DEFINE_BINARY_PROTO_FUZZER(arg) DEFINE_PROTO_FUZZER_IMPL(true, arg)

#define DEFINE_PROTO_FUZZER_IMPL(use_binary, arg) \
static void TestOneProtoInput(arg); \
using FuzzerProtoType = std::remove_const<std::remove_reference< \
std::function<decltype(TestOneProtoInput)>::argument_type>::type>::type; \
DEFINE_CUSTOM_PROTO_MUTATOR_IMPL(use_binary, FuzzerProtoType) \
DEFINE_CUSTOM_PROTO_CROSSOVER_IMPL(use_binary, FuzzerProtoType) \
DEFINE_TEST_ONE_PROTO_INPUT_IMPL(use_binary, FuzzerProtoType) \
DEFINE_POST_PROCESS_PROTO_MUTATION_IMPL(FuzzerProtoType) \
static void TestOneProtoInput(arg)

调用路径为 DEFINE_PROTO_FUZZER -> DEFINE_TEXT_PROTO_FUZZER -> DEFINE_PROTO_FUZZER_IMPL -> DEFINE_TEST_ONE_PROTO_INPUT_IMPL -> LLVMFuzzerTestOneInput -> TestOneProtoInput
最终还是实现了LLVMFuzzerTestOneInput libfuzzer 的入口方法,使用 macro 可以少写不少代码非常方便,从这里就可以看出 fuzz 的 target function 必须和 LLVMFuzzerTestOneInput 的参数类型一致

1
2
3
4
5
// fuzz_target.cc
extern "C" int LLVMFuzzerTestOneInput(const uint8_t *Data, size_t Size) {
DoSomethingInterestingWithMyAPI(Data, Size);
return 0; // Non-zero return values are reserved for future use.
}

libfuzzer_macro.cc

1
2
3
4
5
6
7
8
9
10
bool LoadProtoInput(bool binary, const uint8_t* data, size_t size,
protobuf::Message* input) {
if (GetCache()->LoadIfSame(data, size, input)) return true;
auto result = binary ? ParseBinaryMessage(data, size, input)
: ParseTextMessage(data, size, input);
if (!result) return false;
GetMutator()->Seed(size);
GetMutator()->Fix(input);
return true;
}

LoadProtoInput 返回 true 或者 false,如果解析成功,将调用 TestOneProtoInputDEFINE_PROTO_FUZZER macro 其实就是写 TestOneProtoInput 的实现。

libprotobuf-mutator_fuzzing_learning

这是 github 上一位同行写的学习 libprotobuf-mutator fuzzing 的文章,总体写的不错,但是其中有一些错误的地方,在实践过程中都记录了其中的修改。

Simple protobuf example

文章:https://github.com/bruce30262/libprotobuf-mutator_fuzzing_learning/tree/master/1_simple_protobuf

先写一个简单的 protobuf 文件,test.proto

1
2
3
4
5
6
syntax = "proto2";

message TEST {
required uint32 a = 1;
required string b = 2;
}

使用 protoc 编译 protobuf 文件

1
2
mkdir genfiles  
protoc ./test.proto --cpp_out=./genfiles

将自动生成两个文件

1
2
ls ./genfiles
test.pb.cc test.pb.h

写一个测试程序 test_proto.cc

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
#include "test.pb.h"

#include <bits/stdc++.h>

using std::cin;
using std::cout;
using std::endl;

int main(int argc, char *argv[])
{
TEST t;
t.set_a(101);
t.set_b("testtest");
cout << t.a() << endl;
cout << t.b() << endl;
return 0;
}

写 Makefile 来编译 test_proto.cc

1
2
3
4
5
6
7
8
9
10
11
12
13
CXX=clang++
PB_SRC=test.pb.cc

PROTOBUF_DIR=$(HOME)/code/libprotobuf-mutator/build/external.protobuf/
PROTOBUF_LIB=$(PROTOBUF_DIR)/lib/libprotobufd.a
INC=-I$(PROTOBUF_DIR)/include

test_proto: test_proto.cc $(PB_SRC)
$(CXX) -o $@ $^ $(PROTOBUF_LIB) $(INC)

.PHONY: clean
clean:
rm test_proto

执行 make 后报错 ./test.pb.h:17:2: error: This file was generated by an older version of protoc which is,因为我们在编译的时候使用了 -DLIB_PROTO_MUTATOR_DOWNLOAD_PROTOBUF=ON 下载了新版本的 protobuf,所以出了这个错误。只好使用下载的 protoc 重新生成 test.pb.cctest.pb.h

1
~/code/libprotobuf-mutator/build/external.protobuf/bin/protoc ./test.proto --cpp_out=./genfiles

这次可以成功 make 了,实际执行的命令是

1
2
clang++ -o test_proto test_proto.cc test.pb.cc /home/henices/code/libprotobuf-mutator/build/external.protobuf//lib/libprotobufd.a \
-I/home/henices/code/libprotobuf-mutator/build/external.protobuf//include

运行 ./test_proto, 输出如下

1
2
101
testtest

Combine libprotobuf-mutator with libfuzzer

代码在: https://github.com/bruce30262/libprotobuf-mutator_fuzzing_learning/tree/master/2_libprotobuf_libfuzzer

harness.cc

1
2
3
4
5
6
7
8
9
#include <stdint.h>
#include <stddef.h>

extern "C" int FuzzTEST(const uint8_t *data, size_t size) {
if(data[0] == '\x01') {
__builtin_trap();
}
return 0;
}

其中 FuzzTEST 是我们需要测试的目标函数。

lpm_libfuzz.cc

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
#include "libprotobuf-mutator/src/libfuzzer/libfuzzer_macro.h"
#include "test.pb.h"

#include <bits/stdc++.h>

using std::cin;
using std::cout;
using std::endl;

std::string ProtoToData(const TEST &test_proto) {
std::stringstream all;
const auto &aa = test_proto.a();
const auto &bb = test_proto.b();
all.write((const char*)&aa, sizeof(aa));
if(bb.size() != 0) {
all.write(bb.c_str(), bb.size());
}

std::string res = all.str();
if (bb.size() != 0 && res.size() != 0) {
// set PROTO_FUZZER_DUMP_PATH env to dump the serialized protobuf
if (const char *dump_path = getenv("PROTO_FUZZER_DUMP_PATH")) {
std::ofstream of(dump_path);
of.write(res.data(), res.size());
}
}
return res;
}

extern "C" int FuzzTEST(const uint8_t* data, size_t size); // our customized fuzzing function

DEFINE_PROTO_FUZZER(const TEST &test_proto) {
auto s = ProtoToData(test_proto); // convert protobuf to raw data
FuzzTEST((const uint8_t*)s.data(), s.size()); // fuzz the function
}

在文件的最开头导入 libfuzzer_macro.h, 后面就可以使用一些宏来写代码了,DEFINE_PROTO_FUZZER 是关键的 fuzzer 入口。结构化变异部分 libprotobuf-mutator 已经完成了,需要实现的是一个由 protobuf 转需要的数据类型的函数,如上面的 ProtoToData

代码编译

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
TARGET=lpm_libfuzz
CXX=clang++
CXXFLAGS=-g -fsanitize=fuzzer,address
PB_SRC=test.pb.cc

PROTOBUF_DIR=$(HOME)/code/libprotobuf-mutator/build/external.protobuf/
LPM_DIR=$(HOME)/code/AFL+/external/libprotobuf-mutator
PROTOBUF_LIB=$(PROTOBUF_DIR)/lib/libprotobufd.a
LPM_LIB=$(LPM_DIR)/lib/libprotobuf-mutator-libfuzzer.a $(LPM_DIR)/lib/libprotobuf-mutator.a
INC=-I$(PROTOBUF_DIR)/include -I$(LPM_DIR)/include
DFUZZ=-DLLVMFuzzerTestOneInput=FuzzTEST

all: $(TARGET)

# for testing libprotobuf + libfuzzer
# compile harness first
# then link lpm_libfuzz with harness.o & static libraries
harness.o: harness.cc
$(CXX) $(CXXFLAGS) -c $(DFUZZ) $<

$(TARGET): harness.o $(TARGET).cc
$(CXX) $(CXXFLAGS) -o $@ $^ $(PB_SRC) $(LPM_LIB) $(PROTOBUF_LIB) $(INC) # $(LPM_LIB) must be placed before $(PROTOBUF_LIB)

.PHONY: clean
clean:
rm $(TARGET) *.o

make 后报错,找不到头文件

1
2
3
4
/home/henices/code/AFL+/external/libprotobuf-mutator/include/libprotobuf-mutator/src/libfuzzer/libfuzzer_macro.h:24:10: fatal error: 'port/protobuf.h' file not found
#include "port/protobuf.h"
^~~~~~~~~~~~~~~~~
1 error generated.

port/protobuf.h 来自 https://github.com/google/libprotobuf-mutator/blob/master/port/protobuf.h 修改 Makefile 如下

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
TARGET=lpm_libfuzz
CXX=clang++
CXXFLAGS=-g -fsanitize=fuzzer,address
PB_SRC=test.pb.cc

PROTOBUF_DIR=$(HOME)/code/libprotobuf-mutator/build/external.protobuf/
LPM_DIR=$(HOME)/code/AFL+/external/libprotobuf-mutator
PROTOBUF_LIB=$(PROTOBUF_DIR)/lib/libprotobufd.a
LPM_LIB=$(LPM_DIR)/lib/libprotobuf-mutator-libfuzzer.a $(LPM_DIR)/lib/libprotobuf-mutator.a
INC=-I$(PROTOBUF_DIR)/include -I$(HOME)/code/libprotobuf-mutator/ -I$(LPM_DIR)/include
DFUZZ=-DLLVMFuzzerTestOneInput=FuzzTEST

all: $(TARGET)

# for testing libprotobuf + libfuzzer
# compile harness first
# then link lpm_libfuzz with harness.o & static libraries
harness.o: harness.cc
$(CXX) $(CXXFLAGS) -c $(DFUZZ) $<

$(TARGET): harness.o $(TARGET).cc
$(CXX) $(CXXFLAGS) -o $@ $^ $(PB_SRC) $(LPM_LIB) $(PROTOBUF_LIB) $(INC) # $(LPM_LIB) must be placed before $(PROTOBUF_LIB)

.PHONY: clean
clean:
rm $(TARGET) *.o

修改 makefile 后,可以正常编译通过。经过我们上面的分析,不需要定义 DFUZZ=-DLLVMFuzzerTestOneInput=FuzzTEST 将这行删除掉,同样可以编译通过。

执行 lpm_libfuzz 运行正常

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
➜  ./lpm_libfuzz                                                                                                                                                                                          
INFO: found LLVMFuzzerCustomMutator (0x758a80). Disabling -len_control by default.
INFO: Running with entropic power schedule (0xFF, 100).
INFO: Seed: 331712324
INFO: Loaded 1 modules (434 inline 8-bit counters): 434 [0xa056b8, 0xa0586a),
INFO: Loaded 1 PC tables (434 PCs): 434 [0x939398,0x93aeb8),
INFO: -max_len is not provided; libFuzzer will not generate inputs larger than 4096 bytes
INFO: A corpus is not provided, starting from an empty corpus
#2 INITED cov: 82 ft: 83 corp: 1/1b exec/s: 0 rss: 35Mb
NEW_FUNC[1/41]: 0x75abb0 in TEST::~TEST() /tmp/genfiles/test.pb.cc:116
NEW_FUNC[2/41]: 0x75ca10 in google::protobuf::UnknownFieldSet* google::protobuf::internal::InternalMetadata::mutable_unknown_fields<google::protobuf::UnknownFieldSet>() /home/henices/code/libprotobuf-mut
ator/build/external.protobuf//include/google/protobuf/metadata_lite.h:117
#3 NEW cov: 135 ft: 159 corp: 2/12b lim: 4096 exec/s: 0 rss: 36Mb L: 11/11 MS: 1 CustomCrossOver-
#4 NEW cov: 138 ft: 162 corp: 3/201b lim: 4096 exec/s: 0 rss: 36Mb L: 189/189 MS: 2 InsertRepeatedBytes-Custom-
#38 REDUCE cov: 138 ft: 162 corp: 3/127b lim: 4096 exec/s: 0 rss: 37Mb L: 115/115 MS: 5 CustomCrossOver-CustomCrossOver-Custom-InsertRepeatedBytes-Custom-
#60 REDUCE cov: 138 ft: 162 corp: 3/124b lim: 4096 exec/s: 0 rss: 37Mb L: 112/112 MS: 4 CustomCrossOver-ShuffleBytes-ChangeByte-Custom-
#98 REDUCE cov: 138 ft: 162 corp: 3/56b lim: 4096 exec/s: 0 rss: 37Mb L: 44/44 MS: 5 CrossOver-Custom-Custom-CMP-Custom- DE: "~\xff\xff\xff\xff\xff\xff\xff"-
#144 REDUCE cov: 138 ft: 162 corp: 3/53b lim: 4096 exec/s: 0 rss: 37Mb L: 41/41 MS: 2 CrossOver-Custom-
#182 REDUCE cov: 138 ft: 162 corp: 3/25b lim: 4096 exec/s: 0 rss: 37Mb L: 13/13 MS: 5 ChangeBit-Custom-Custom-InsertByte-Custom-
#324 REDUCE cov: 138 ft: 162 corp: 3/24b lim: 4096 exec/s: 0 rss: 38Mb L: 12/12 MS: 3 CustomCrossOver-EraseBytes-Custom-

Handling input from AFL++ in our custom mutator

https://github.com/bruce30262/libprotobuf-mutator_fuzzing_learning/tree/master/5_libprotobuf_aflpp_custom_mutator_input

主要内容是将 libprotobuf-mutator 和 afl++ 结合起来,使用的 afl++ 的 custom mutator,值得一提的是在这个例子里,需要使用 -fPIC 参数编译 libprotobuf-mutator。

1
2
3
4
cmake .. -GNinja -DCMAKE_C_COMPILER=clang -DLIB_PROTO_MUTATOR_DOWNLOAD_PROTOBUF=ON \
-DCMAKE_INSTALL_PREFIX=/home/henices/code/AFL+/external/libprotobuf-mutator \
-DCMAKE_CXX_COMPILER=clang++ -DCMAKE_BUILD_TYPE=Debug \
-DCMAKE_C_FLAGS="-fPIC" -DCMAKE_CXX_FLAGS="-fPIC"

主要的步骤在 readme.md 中已经介绍得比较清楚了。

  • lpm_aflpp_custom_mutator_input.cc 是 afl++ 的 custom mutator shared library
    • 解析输入数据(testcase buffer)并将其转成 TEST protobuf message
    • 使用 libprotobuf-mutator 变异 TEST protobuf message
    • 注册一个 PostProcessor 处理变异后的 TEST protobuf message (非必要步骤)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
extern "C" size_t afl_custom_fuzz(MyMutator *mutator, // return value from afl_custom_init
uint8_t *buf, size_t buf_size, // input data to be mutated
uint8_t **out_buf, // output buffer
uint8_t *add_buf, size_t add_buf_size, // add_buf can be NULL
size_t max_size) {
// This function can be named either "afl_custom_fuzz" or "afl_custom_mutator"
// A simple test shows that "buf" will be the content of the current test case
// "add_buf" will be the next test case ( from AFL++'s input queue )

TEST input;
// parse input data to TEST
// Notice that input data should be a serialized protobuf data
// Check ./in/ii and test_protobuf_serializer for more detail
bool parse_ok = input.ParseFromArray(buf, buf_size);
if(!parse_ok) {
// Invalid serialize protobuf data. Don't mutate.
// Return a dummy buffer. Also mutated_size = 0
static uint8_t *dummy = new uint8_t[10]; // dummy buffer with no data
*out_buf = dummy;
return 0;
}
// mutate the protobuf
mutator->Mutate(&input, max_size);

// Convert protobuf to raw data
const TEST *p = &input;
std::string s = ProtoToData(*p);
// Copy to a new buffer ( mutated_out )
size_t mutated_size = s.size() <= max_size ? s.size() : max_size; // check if raw data's size is larger than max_size
uint8_t *mutated_out = new uint8_t[mutated_size+1];
memcpy(mutated_out, s.c_str(), mutated_size); // copy the mutated data
// Assign the mutated data and return mutated_size
*out_buf = mutated_out;
return mutated_size;
}

mutator->Mutate(&input, max_size); 为真正起作用的核心代码

  • lpm_aflpp_custom_mutator_input.h 继承了protobuf_mutator::Mutator, 可以使用 libprotobuf-mutator 的 Mutate 方法
1
2
3
4
5
6
7
#include "libprotobuf-mutator/src/mutator.h"
#include "test.pb.h"

#include <bits/stdc++.h>

class MyMutator : public protobuf_mutator::Mutator {
};
  • test_proto_serializer.cc

    • 用于生成一条序列化的 TEST protobuf message,可以作为 fuzz 的初始化 testcase 使用
  • vuln.c 漏洞测试程序

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
#include <stdio.h>
#include <string.h>
#include <math.h>
#include <stdlib.h>
#include <unistd.h>

int main(int argc, char *argv[])
{
char str[100]={};
read(0, str, 100);
int *ptr = NULL;
if( str[0] == '\x02' || str[0] == '\xe8') {
*ptr = 123;
}
return 0;
}

漏洞测试程序比较简单,只要第一个字节是 0xe8 或者 0x02 即可,libprotobuf-mutator 的变异在这个例子里效率并不高,所以需要使用 PostProcessor 来优化变异。

  • Makefile
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
TARGET=lpm_aflpp_custom_mutator_input
CXX=clang++-11
AFLCC=$(HOME)/AFLplusplus/afl-gcc
PB_SRC=test.pb.cc

PROTOBUF_DIR=$(HOME)/libprotobuf-mutator/build/external.protobuf
PROTOBUF_LIB=$(PROTOBUF_DIR)/lib/libprotobufd.a

LPM_DIR=$(HOME)/libprotobuf-mutator
LPM_LIB=$(LPM_DIR)/build/src/libfuzzer/libprotobuf-mutator-libfuzzer.a $(LPM_DIR)/build/src/libprotobuf-mutator.a

INC=-I$(PROTOBUF_DIR)/include -I$(LPM_DIR)

all: $(TARGET).so

$(TARGET).so: $(TARGET).cc $(PB_SRC)
$(CXX) -fPIC -c $^ $(INC)
$(CXX) -shared -Wall -O3 -o $@ *.o $(LPM_LIB) $(PROTOBUF_LIB)

vuln: vuln.c
$(AFLCC) -o $@ $^

test_proto_serializer: test_proto_serializer.cc $(PB_SRC)
$(CXX) -o $@ $^ $(PROTOBUF_LIB) $(INC)

.PHONY: clean
clean:
rm *.so *.o vuln test_proto_serializer

Makefile 有瑕疵,这个章节的内容和 libfuzzer 没有关系,不需要链接 libprotobuf-mutator-libfuzzer.a

参考资料

https://developers.google.com/protocol-buffers/docs/cpptutorial
https://github.com/google/fuzzing/blob/master/docs/structure-aware-fuzzing.md
https://github.com/google/libprotobuf-mutator/
https://llvm.org/docs/LibFuzzer.html
https://github.com/bruce30262/libprotobuf-mutator_fuzzing_learning