Fixing random Quartus II crashes
Quartus II freezes
If like me you are writing HDL for an older Intel (Altera) familiy FPGA, like let’s say a Cyclone II, you are stuck with an older version of Quartus. The last version of Quartus that supports the Cyclone II family is Quartus II 13.1.0.162. Unfortunately, Quartus II has been unmaintained for a long time. Unfortunately, Quartus II may crash a lot.
I use an old Core-2-Duo-based machine that is more than 10 years old. My lab is in my basement and I use VNC in order to use Quartus from the comfort of my living room. Sometimes, Quartus would freeze without warning, leaving me with no other choice but to kill the application. One freeze every hour could be manageable. But at some point, Quartus would feeze every 3 minutes which became simply unusable. The freeze could happen anywhere, anytime: inside the RTL viewer, while synthesizing, you name it. I needed a solution other than jumping to a newer FPGA family.
Isolate the problem
My first idea at this point was to fire up gdb to try and catch an occurence of the freeze,
with the hope of getting a backtrace. Since the bug is easily reproducible, this shouldn’t
be too hard. Simply attach to the Quartus process by giving the correct PID to gdb.
Once Quartus freezes, just hit ^C to pause the application and type bt.
[user@quartus linux64]$ gdb -p 3470
GNU gdb (GDB) 12.1
Copyright (C) 2022 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-pc-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word".
Attaching to process 3470
[New LWP 3471]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/usr/lib/libthread_db.so.1".
0x00007f38341058eb in sched_yield () from /usr/lib/libc.so.6
(gdb) bt
#0 0x00007ff5e01058eb in sched_yield () from /usr/lib/libc.so.6
#1 0x00007ff5c3a096a5 in rml::internal::BackRefIdx::newBackRef(bool) () from /opt/quartus_13.0.1/quartus/linux64/libtbbmalloc.so.2
#2 0x00007ff5c3a08f1b in rml::internal::mallocLargeObject(rml::internal::ExtMemoryPool*, unsigned long, unsigned long, bool) () from /opt/quartus_13.0.1/quartus/linux64/libtbbmalloc.so.2
#3 0x00007ff5c3a05ff6 in scalable_malloc () from /opt/quartus_13.0.1/quartus/linux64/libtbbmalloc.so.2
#4 0x00007ff5c4600fd9 in malloc () from /opt/quartus_13.0.1/quartus/linux64/libtbbmalloc_proxy.so.2
#5 0x00007ff5c3a068a8 in rml::internal::ErrnoPreservingMalloc(unsigned long) () from /opt/quartus_13.0.1/quartus/linux64/libtbbmalloc.so.2
#6 0x00007ff5c3a094d6 in rml::internal::BackRefMaster::findFreeBlock() () from /opt/quartus_13.0.1/quartus/linux64/libtbbmalloc.so.2
#7 0x00007ff5c3a095ef in rml::internal::BackRefIdx::newBackRef(bool) () from /opt/quartus_13.0.1/quartus/linux64/libtbbmalloc.so.2
#8 0x00007ff5c3a08f1b in rml::internal::mallocLargeObject(rml::internal::ExtMemoryPool*, unsigned long, unsigned long, bool) () from /opt/quartus_13.0.1/quartus/linux64/libtbbmalloc.so.2
#9 0x00007ff5c3a05ff6 in scalable_malloc () from /opt/quartus_13.0.1/quartus/linux64/libtbbmalloc.so.2
#10 0x00007ff5c4600fd9 in malloc () from /opt/quartus_13.0.1/quartus/linux64/libtbbmalloc_proxy.so.2
#11 0x00007ff5c9bc94f9 in ?? () from /opt/quartus_13.0.1/quartus/linux64/libQtGui.so.4
#12 0x00007ff5c9be5b64 in ?? () from /opt/quartus_13.0.1/quartus/linux64/libQtGui.so.4
#13 0x00007ff5c9bc2b0a in ?? () from /opt/quartus_13.0.1/quartus/linux64/libQtGui.so.4
#14 0x00007ff5c9bcbc50 in ?? () from /opt/quartus_13.0.1/quartus/linux64/libQtGui.so.4
#15 0x00007ff5c9bcfd28 in ?? () from /opt/quartus_13.0.1/quartus/linux64/libQtGui.so.4
#16 0x00007ff5c9b584c3 in QPainter::drawTextItem(QPointF const&, QTextItem const&) () from /opt/quartus_13.0.1/quartus/linux64/libQtGui.so.4
#17 0x00007ff5c9c640d5 in QTextLine::draw(QPainter*, QPointF const&, QTextLayout::FormatRange const*) const () from /opt/quartus_13.0.1/quartus/linux64/libQtGui.so.4
#18 0x00007ff5c9c6786f in QTextLayout::draw(QPainter*, QPointF const&, QVector<QTextLayout::FormatRange> const&, QRectF const&) const () from /opt/quartus_13.0.1/quartus/linux64/libQtGui.so.4
#19 0x00007ff5c9ca339e in ?? () from /opt/quartus_13.0.1/quartus/linux64/libQtGui.so.4
#20 0x00007ff5c9ca726a in ?? () from /opt/quartus_13.0.1/quartus/linux64/libQtGui.so.4
#21 0x00007ff5c9ca6bc7 in ?? () from /opt/quartus_13.0.1/quartus/linux64/libQtGui.so.4
#22 0x00007ff5c9cb03c6 in ?? () from /opt/quartus_13.0.1/quartus/linux64/libQtGui.so.4
#23 0x00007ff5c9c793fe in QTextDocument::drawContents(QPainter*, QRectF const&) () from /opt/quartus_13.0.1/quartus/linux64/libQtGui.so.4
#24 0x00007ff5988293e1 in MSWQ_DELEGATE::paint(QPainter*, QStyleOptionViewItem const&, QModelIndex const&) const () from /opt/quartus_13.0.1/quartus/linux64/libresr_mswq.so
#25 0x00007ff5c9f4db1b in QTreeView::drawRow(QPainter*, QStyleOptionViewItem const&, QModelIndex const&) const () from /opt/quartus_13.0.1/quartus/linux64/libQtGui.so.4
#26 0x00007ff5d4e90a14 in AFCQ_TREE_VIEW::drawRow(QPainter*, QStyleOptionViewItem const&, QModelIndex const&) const () from /opt/quartus_13.0.1/quartus/linux64/libgcl_afcq.so
#27 0x00007ff5c9f46130 in QTreeView::drawTree(QPainter*, QRegion const&) const () from /opt/quartus_13.0.1/quartus/linux64/libQtGui.so.4
#28 0x00007ff5c9f46ab3 in QTreeView::paintEvent(QPaintEvent*) () from /opt/quartus_13.0.1/quartus/linux64/libQtGui.so.4
#29 0x00007ff5c9a4f522 in QWidget::event(QEvent*) () from /opt/quartus_13.0.1/quartus/linux64/libQtGui.so.4
#30 0x00007ff5c9df1cb6 in QFrame::event(QEvent*) () from /opt/quartus_13.0.1/quartus/linux64/libQtGui.so.4
#31 0x00007ff5c9f01a63 in QAbstractItemView::viewportEvent(QEvent*) () from /opt/quartus_13.0.1/quartus/linux64/libQtGui.so.4
#32 0x00007ff5c9f47df5 in QTreeView::viewportEvent(QEvent*) () from /opt/quartus_13.0.1/quartus/linux64/libQtGui.so.4
#33 0x00007ff5c938b407 in QCoreApplicationPrivate::sendThroughObjectEventFilters(QObject*, QEvent*) () from /opt/quartus_13.0.1/quartus/linux64/libQtCore.so.4
#34 0x00007ff5c99fd8f1 in QApplicationPrivate::notify_helper(QObject*, QEvent*) () from /opt/quartus_13.0.1/quartus/linux64/libQtGui.so.4
#35 0x00007ff5c9a024df in QApplication::notify(QObject*, QEvent*) () from /opt/quartus_13.0.1/quartus/linux64/libQtGui.so.4
#36 0x00007ff5c938b27c in QCoreApplication::notifyInternal(QObject*, QEvent*) () from /opt/quartus_13.0.1/quartus/linux64/libQtCore.so.4
#37 0x00007ff5c9a4b0e8 in QWidgetPrivate::drawWidget(QPaintDevice*, QRegion const&, QPoint const&, int, QPainter*, QWidgetBackingStore*) () from /opt/quartus_13.0.1/quartus/linux64/libQtGui.so.4
#38 0x00007ff5c9c1a700 in ?? () from /opt/quartus_13.0.1/quartus/linux64/libQtGui.so.4
#39 0x00007ff5c9a41780 in QWidgetPrivate::syncBackingStore() () from /opt/quartus_13.0.1/quartus/linux64/libQtGui.so.4
#40 0x00007ff5c9a4fa22 in QWidget::event(QEvent*) () from /opt/quartus_13.0.1/quartus/linux64/libQtGui.so.4
#41 0x00007ff5c9e0bbeb in QMainWindow::event(QEvent*) () from /opt/quartus_13.0.1/quartus/linux64/libQtGui.so.4
#42 0x00007ff5c99fd924 in QApplicationPrivate::notify_helper(QObject*, QEvent*) () from /opt/quartus_13.0.1/quartus/linux64/libQtGui.so.4
#43 0x00007ff5c9a024df in QApplication::notify(QObject*, QEvent*) () from /opt/quartus_13.0.1/quartus/linux64/libQtGui.so.4
#44 0x00007ff5c938b27c in QCoreApplication::notifyInternal(QObject*, QEvent*) () from /opt/quartus_13.0.1/quartus/linux64/libQtCore.so.4
#45 0x00007ff5c938e8a8 in QCoreApplicationPrivate::sendPostedEvents(QObject*, int, QThreadData*) () from /opt/quartus_13.0.1/quartus/linux64/libQtCore.so.4
#46 0x00007ff5c93bbfb0 in QEventDispatcherUNIX::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) () from /opt/quartus_13.0.1/quartus/linux64/libQtCore.so.4
#47 0x00007ff5c9aa38b2 in ?? () from /opt/quartus_13.0.1/quartus/linux64/libQtGui.so.4
#48 0x00007ff5c938eebf in QCoreApplication::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) () from /opt/quartus_13.0.1/quartus/linux64/libQtCore.so.4
#49 0x00007ff5e245d1c5 in qui_call_app_process_event () from /opt/quartus_13.0.1/quartus/linux64/libsys_qui.so
#50 0x00007ff5ca607346 in thr_qt_process_events () from /opt/quartus_13.0.1/quartus/linux64/libccl_thr.so
#51 0x00007ff5dde1ab35 in flow_process_events() () from /opt/quartus_13.0.1/quartus/linux64/libsys_flow.so
#52 0x00007ff5dde36fa4 in flow_call_cmd(FLOW_FACADE*, char const*) () from /opt/quartus_13.0.1/quartus/linux64/libsys_flow.so
#53 0x00007ff5dde41a39 in flow_generic_call(FLOW_FACADE*, char const*, char const*) () from /opt/quartus_13.0.1/quartus/linux64/libsys_flow.so
#54 0x00007ff5dde41f4b in flow_do_fit_portion(FLOW_FACADE*, HDB_CMP_ACTION_PT_INSTANCE::SMART_ACTION) () from /opt/quartus_13.0.1/quartus/linux64/libsys_flow.so
#55 0x00007ff5dde4c4a1 in flow_full_compile_or_simulate(FLOW_FACADE*, FLOW_INFO, bool, _Dinkum_std::vector<_Dinkum_std::basic_string<char, _Dinkum_std::char_traits<char>, MEM_STL_ALLOCATOR<char> >, MEM_STL_ALLOCATOR<_Dinkum_std::basic_string<char, _Dinkum_std::char_traits<char>, MEM_STL_ALLOCATOR<char> > > >*) () from /opt/quartus_13.0.1/quartus/linux64/libsys_flow.so
#56 0x00007ff5dde4d4a3 in flow_compile(FLOW_FACADE*, FLOW_INFO) () from /opt/quartus_13.0.1/quartus/linux64/libsys_flow.so
#57 0x00007ff5dde4e25e in flow_execute_custom () from /opt/quartus_13.0.1/quartus/linux64/libsys_flow.so
#58 0x00007ff5dde4ee05 in flow_execute () from /opt/quartus_13.0.1/quartus/linux64/libsys_flow.so
#59 0x00007ff5e24c63ca in QUI_APP::on_start_flow(int, QString) () from /opt/quartus_13.0.1/quartus/linux64/libsys_qui.so
#60 0x00007ff5e248680a in QUI_APP::qt_static_metacall(QObject*, QMetaObject::Call, int, void**) () from /opt/quartus_13.0.1/quartus/linux64/libsys_qui.so
#61 0x00007ff5c93a0056 in QObject::event(QEvent*) () from /opt/quartus_13.0.1/quartus/linux64/libQtCore.so.4
#62 0x00007ff5c99fff26 in QApplication::event(QEvent*) () from /opt/quartus_13.0.1/quartus/linux64/libQtGui.so.4
#63 0x00007ff5c99fd924 in QApplicationPrivate::notify_helper(QObject*, QEvent*) () from /opt/quartus_13.0.1/quartus/linux64/libQtGui.so.4
#64 0x00007ff5c9a024df in QApplication::notify(QObject*, QEvent*) () from /opt/quartus_13.0.1/quartus/linux64/libQtGui.so.4
#65 0x00007ff5c938b27c in QCoreApplication::notifyInternal(QObject*, QEvent*) () from /opt/quartus_13.0.1/quartus/linux64/libQtCore.so.4
#66 0x00007ff5c938e8a8 in QCoreApplicationPrivate::sendPostedEvents(QObject*, int, QThreadData*) () from /opt/quartus_13.0.1/quartus/linux64/libQtCore.so.4
#67 0x00007ff5c9aa36ca in ?? () from /opt/quartus_13.0.1/quartus/linux64/libQtGui.so.4
#68 0x00007ff5c938a062 in QEventLoop::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) () from /opt/quartus_13.0.1/quartus/linux64/libQtCore.so.4
#69 0x00007ff5c938a31d in QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>) () from /opt/quartus_13.0.1/quartus/linux64/libQtCore.so.4
#70 0x00007ff5c938ed4b in QCoreApplication::exec() () from /opt/quartus_13.0.1/quartus/linux64/libQtCore.so.4
#71 0x0000000000407eb1 in qgq_main(int, char const**) ()
#72 0x00007ff5cf8361b0 in msg_main_thread(void*) () from /opt/quartus_13.0.1/quartus/linux64/libccl_msg.so
#73 0x00007ff5ca607fac in thr_final_wrapper () from /opt/quartus_13.0.1/quartus/linux64/libccl_thr.so
#74 0x00007ff5cf836d55 in msg_thread_wrapper(void* (*)(void*), void*) () from /opt/quartus_13.0.1/quartus/linux64/libccl_msg.so
#75 0x0000000000416fd5 in mem_thread_wrapper(void* (*)(void*), void*) ()
#76 0x00007ff5cb20f408 in err_thread_wrapper(void* (*)(void*), void*) () from /opt/quartus_13.0.1/quartus/linux64/libccl_err.so
#77 0x00007ff5ca60838c in thr_thread_wrapper () from /opt/quartus_13.0.1/quartus/linux64/libccl_thr.so
#78 0x00007ff5cf848f4d in msg_exe_main(int, char const**, int (*)(int, char const**)) () from /opt/quartus_13.0.1/quartus/linux64/libccl_msg.so
#79 0x0000000000407f75 in main ()
Since the crash may happen from anywhere, let’s try a second time with the hope of finding
similarities between both traces. Reproducing indeed shows a common denominator: libtbb_malloc.so.
#0 0x00007f38341058eb in sched_yield () from /usr/lib/libc.so.6
#1 0x00007f3817a096a5 in rml::internal::BackRefIdx::newBackRef(bool) () from /opt/quartus_13.0.1/quartus/linux64/libtbbmalloc.so.2
#2 0x00007f3817a08f1b in rml::internal::mallocLargeObject(rml::internal::ExtMemoryPool*, unsigned long, unsigned long, bool) () from /opt/quartus_13.0.1/quartus/linux64/libtbbmalloc.so.2
#3 0x00007f3817a05ff6 in scalable_malloc () from /opt/quartus_13.0.1/quartus/linux64/libtbbmalloc.so.2
#4 0x00007f3818600fd9 in malloc () from /opt/quartus_13.0.1/quartus/linux64/libtbbmalloc_proxy.so.2
#5 0x00007f3817a068a8 in rml::internal::ErrnoPreservingMalloc(unsigned long) () from /opt/quartus_13.0.1/quartus/linux64/libtbbmalloc.so.2
#6 0x00007f3817a094d6 in rml::internal::BackRefMaster::findFreeBlock() () from /opt/quartus_13.0.1/quartus/linux64/libtbbmalloc.so.2
#7 0x00007f3817a095ef in rml::internal::BackRefIdx::newBackRef(bool) () from /opt/quartus_13.0.1/quartus/linux64/libtbbmalloc.so.2
#8 0x00007f3817a08f1b in rml::internal::mallocLargeObject(rml::internal::ExtMemoryPool*, unsigned long, unsigned long, bool) () from /opt/quartus_13.0.1/quartus/linux64/libtbbmalloc.so.2
#9 0x00007f3817a05db8 in rml::internal::reallocAligned(rml::internal::MemoryPool*, void*, unsigned long, unsigned long) () from /opt/quartus_13.0.1/quartus/linux64/libtbbmalloc.so.2
#10 0x00007f3817a063d5 in safer_scalable_realloc () from /opt/quartus_13.0.1/quartus/linux64/libtbbmalloc.so.2
#11 0x00007f3818601069 in realloc () from /opt/quartus_13.0.1/quartus/linux64/libtbbmalloc_proxy.so.2
Therefore we have good reasons to believe that this is caused by a bug inside Intel’s Thread Building Block
library, specifically inside libtbbmalloc.so.2 shipped with Quartus.
Fix the problem
We have multiple options to fix the problem. Some may be easy, others may take time. Our goal is to spend the least amount of time possible. Here’s what I tried, in order.
Use your distro’s tbb package (failed)
The path of least resistance is probably to try a current, up-to-date version of the library. Quartus 13 being quite old, this is a long shot. But it has virtually zero cost, so let’s try it:
pacman -S tbb
# Rename the library so that the one from the system gets loaded instead
mv /opt/quartus_13.0.1/quartus/linux64/libtbb.so.2{,.bkp}
Unfortunately, Quartus doesn’t start:
[user@quartus linux64]$ /home/user/dev/fpga/quartus/start.sh
/opt/quartus_13.0.1/quartus/linux64/quartus: /opt/quartus_13.0.1/quartus/linux64/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by /opt/quartus_13.0.1/quartus/linux64/libtbbmalloc_proxy.so.2)
Quartus 13 ships its own ancient stdlibc++.so and libtbb doesn’t like it. With that in mind, what if we build libtbb by ourselves against the same version of Quartus’ standard C library?
Build libtbb with an older glibc (failed)
Fortunately, Intel’s Thread Building Block library is open source, meaning that we can build it as we wish. libtbb needs to be linked against the correct version if the standard C library. We thus need to find an old toolchain bundled with the correct glibc version. Ultimately, we could build our own but building a toolchain takes a lot of time. Surely we can find a prebuilt one somewhere on the web?
Using objdump, we can peek into the comment section of the tbb library to gather information about the compiler that was used, refer to https://unix.stackexchange.com/a/722.
[user@quartus linux64]$ objdump -s --section .comment libtbbmalloc.so.2.bkp
libtbbmalloc.so.2.bkp: file format elf64-x86-64
Contents of section .comment:
0000 00474343 3a202847 4e552920 342e312e .GCC: (GNU) 4.1.
0010 32203230 30383037 30342028 52656420 2 20080704 (Red
0020 48617420 342e312e 322d3532 29000047 Hat 4.1.2-52)..G
0030 43433a20 28474e55 2920342e 352e3300 CC: (GNU) 4.5.3.
0040 00474343 3a202847 4e552920 342e352e .GCC: (GNU) 4.5.
0050 33000047 43433a20 28474e55 2920342e 3..GCC: (GNU) 4.
0060 352e3300 00474343 3a202847 4e552920 5.3..GCC: (GNU)
0070 342e352e 33000047 43433a20 28474e55 4.5.3..GCC: (GNU
0080 2920342e 352e3300 00474343 3a202847 ) 4.5.3..GCC: (G
0090 4e552920 342e352e 33000047 43433a20 NU) 4.5.3..GCC:
00a0 28474e55 2920342e 352e3300 00474343 (GNU) 4.5.3..GCC
00b0 3a202847 4e552920 342e352e 33000047 : (GNU) 4.5.3..G
00c0 43433a20 28474e55 2920342e 312e3220 CC: (GNU) 4.1.2
00d0 32303038 30373034 20285265 64204861 20080704 (Red Ha
00e0 7420342e 312e322d 35322900 t 4.1.2-52).
In an ideal world, we need to build tbb with gcc 4.2. Practically, the Arch Linux Historical Archive
contains older gcc packages down to version gcc 4.8.1.
Let’s try that. Trying to build the tbb package by running makepkg fails to recognise some compile switches.
At this point, I realize that building tbb with an older compiler may very well be
a dead-end and very time-consuming. It’s time to choose a different approach.
Find an older prebuilt version of libtbb (success!)
Using objdump, we can even find out what libtbb version is used by Quartus 13.
[user@quartus linux64]$ objdump -s libtbbmalloc.so.2.bkp
libtbbmalloc.so.2.bkp: file format elf64-x86-64
[...]
Contents of section .data:
20fe20 20fe2000 00000000 70b00000 00000000 . .....p.......
20fe30 30b00000 00000000 f0af0000 00000000 0...............
[...]
2100e0 00544242 3a205645 5253494f 4e090934 .TBB: VERSION..4
2100f0 2e300a54 42423a20 494e5445 52464143 .0.TBB: INTERFAC
210100 45205645 5253494f 4e093630 30320a54 E VERSION.6002.T
210110 42423a20 4255494c 445f4441 54450909 BB: BUILD_DATE..
210120 54687520 4a756e20 31332030 313a3339 Thu Jun 13 01:39
210130 3a353420 55544320 32303133 0a544242 :54 UTC 2013.TBB
210140 3a204255 494c445f 484f5354 0909736a : BUILD_HOST..sj
210150 2d737762 6c642d6c 6e783230 20287838 -swbld-lnx20 (x8
210160 365f3634 290a5442 423a2042 55494c44 6_64).TBB: BUILD
210170 5f4f5309 0943656e 744f5320 72656c65 _OS..CentOS rele
210180 61736520 352e3620 2846696e 616c290a ase 5.6 (Final).
210190 5442423a 20425549 4c445f4b 45524e45 TBB: BUILD_KERNE
2101a0 4c094c69 6e757820 322e362e 31382d32 L.Linux 2.6.18-2
2101b0 33382e65 6c352023 3120534d 50205468 38.el5 #1 SMP Th
2101c0 75204a61 6e203133 2031353a 35313a31 u Jan 13 15:51:1
2101d0 35204553 54203230 31310a54 42423a20 5 EST 2011.TBB:
2101e0 4255494c 445f4743 43090967 63632076 BUILD_GCC..gcc v
2101f0 65727369 6f6e2034 2e352e33 20284743 ersion 4.5.3 (GC
210200 4329200a 5442423a 20425549 4c445f47 C) .TBB: BUILD_G
210210 4c494243 09322e35 0a544242 3a204255 LIBC.2.5.TBB: BU
210220 494c445f 4c440909 0a544242 3a204255 ILD_LD...TBB: BU
210230 494c445f 54415247 45540969 6e74656c ILD_TARGET.intel
210240 3634206f 6e206363 342e352e 335f6c69 64 on cc4.5.3_li
210250 6263322e 355f6b65 726e656c 322e362e bc2.5_kernel2.6.
210260 31380a54 42423a20 4255494c 445f434f 18.TBB: BUILD_CO
210270 4d4d414e 4409672b 2b202d44 444f5f49 MMAND.g++ -DDO_I
210280 54545f4e 4f544946 59202d4f 32202d44 TT_NOTIFY -O2 -D
210290 5553455f 50544852 45414420 2d6d3634 USE_PTHREAD -m64
2102a0 202d6650 4943202d 445f5f54 42425f42 -fPIC -D__TBB_B
2102b0 55494c44 3d31202d 57616c6c 202d576e UILD=1 -Wall -Wn
2102c0 6f2d7061 72656e74 68657365 73202d57 o-parentheses -W
2102d0 6e6f2d6e 6f6e2d76 69727475 616c2d64 no-non-virtual-d
2102e0 746f7220 2d492e2e 2f2e2e2f 73726320 tor -I../../src
2102f0 2d492e2e 2f2e2e2f 7372632f 726d6c2f -I../../src/rml/
210300 696e636c 75646520 2d492e2e 2f2e2e2f include -I../../
210310 696e636c 7564650a 5442423a 20544242 include.TBB: TBB
210320 5f555345 5f444542 55470930 0a544242 _USE_DEBUG.0.TBB
210330 3a205442 425f5553 455f4153 53455254 : TBB_USE_ASSERT
210340 09300a54 42423a20 444f5f49 54545f4e .0.TBB: DO_ITT_N
210350 4f544946 5909310a 00000000 OTIFY.1.....
Quartus 13 uses libtbb 4.0, let’s see if we can find mentions of this crash on the web. In fact,
changelogs of older versions are available here: https://abi-laboratory.pro/index.php?view=changelog&l=tbb&v=4.0.5.
We can see that TBB 4.0 Update 2 fixed a race condition in the TBB scalable allocator. This is something that sounds
very familiar: remember the appearance of the safer_scalable_realloc() function in the backtrace at the
beginning of this article.
TBB 4.0 Update 2 commercial-aligned release
Changes (w.r.t. TBB 4.0 Update 1 commercial-aligned release):
- concurrent_bounded_queue now has an abort() operation that releases
threads involved in pending push or pop operations. The released
threads will receive a tbb::user_abort exception.
- Added Community Preview Feature: concurrent_lru_cache container,
a concurrent implementation of LRU (least-recently-used) cache.
Bugs fixed:
- fixed a race condition in the TBB scalable allocator.
- concurrent_queue counter wraparound bug was fixed, which occurred when
the number of push and pop operations exceeded ~>4 billion on IA32.
- fixed races in the TBB scheduler that could put workers asleep too
early, especially in presense of affinitized tasks.
And as it turns out, we can find a prebuilt version of libtbb 4.2 in an old CentOS tbb package!
Recent enough to have our bug fixed and old enough to run with Quartus:
https://vault.centos.org/7.5.1804/os/x86_64/Packages/tbb-4.1-9.20130314.el7.x86_64.rpm
Using rmpextract, we can copy (or symlink) the libraries into the Quartus linux64 folder.
After hours of use, not a single freeze :).
Summary
In order to fix Quartus II, do the following. This assumes that you’re running Quartus on a 64-bit machine:
mkdir /tmp/tbb && cd /tmp/tbb
wget https://vault.centos.org/7.5.1804/os/x86_64/Packages/tbb-4.1-9.20130314.el7.x86_64.rpm
rpmextract.sh tbb-4.1-9.20130314.el7.x86_64.rpm
# set your Quartus installation path accordingly
cp usr/lib64/* /opt/quartus_13.0.1/quartus/linux64/
Here is the md5sum of the CentOS files that I tested (and that solved the issue on my machine):
[user@quartus tbb]$ md5sum usr/lib64/*
38b9d27441217c96315808b352782c70 usr/lib64/libtbbmalloc_proxy.so.2
9753c4de355d74f895f024f3939a20f4 usr/lib64/libtbbmalloc.so.2
0eb81ebf068d599cc944e65eb5de10c0 usr/lib64/libtbb.so.2
And if it helps, this is the command that I use in order to run Quartus 13.0.1 on my system (otherwise Quartus attempts to load libraries from my system and fails to start):
LM_LICENSE_FILE=/home/user/dev/fpga/quartus/license_13.dat LD_LIBRARY_PATH=/opt/quartus_13.0.1/quartus/linux64 PATH=$PATH:/opt/quartus_13.0.1/quartus/linux64/ /opt/quartus_13.0.1/quartus/linux64/quartus