<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><meta http-equiv=Content-Type content="text/html; charset=utf-8"><meta name=Generator content="Microsoft Word 12 (filtered medium)"><!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p
{mso-style-priority:99;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
span.apple-style-span
{mso-style-name:apple-style-span;}
span.EmailStyle19
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--></head><body lang=EN-US link=blue vlink=purple><div class=WordSection1><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>No. In fact, I ran 250,000 pages yesterday on my laptop (50,000, 5 page documents). It took about 1.5 hours without any issues whatsoever. We run such productions all the time, from small jobs like your 5000 pages to as large as 1,000,000+pages. I would suggest that it is not in XEP but within your own threading code. My laptop has Winmasse installed and is configured with 4 XEP processes running on four ports (as my machine is a quad-core CPU and I wanted to maximize the load across the whole CPU). The 4 processes each run in their own JVM with minimal (256MB memory each).<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Are you using something like EnMasse (or WinMasse) on windows that already handles load balancing across a bank of XEP Engines? Or is this something you are writing yourself?<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>We have customers that have been running EnMasse/WinMasse on machines … sometimes for over a year … without doing or touching anything, producing 50,000,000+ pages – no failures, no memory leaks.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>We also have VDPMill for very large production runs which has both a splitter and merger. It can split incoming XML files into chunks (for logically structured things like a batch of invoices), spread the rendering across multiple XEP engines running in separate JVMs within cores/CPUs/even different machines, and optionally combine output into a single file when formatting to PDF/PS/AFP. Single print files with over 500,000 pages have been created with this application (which sits on top of EnMasse).<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Contact me to discuss how you should integrate something like our own pre-written balancer for such applications.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Kevin Brown<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>RenderX<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><div style='border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in'><p class=MsoNormal><b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'>From:</span></b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'> xep-support-bounces@renderx.com [mailto:xep-support-bounces@renderx.com] <b>On Behalf Of </b>Justin Lipton<br><b>Sent:</b> Monday, August 27, 2012 12:58 PM<br><b>To:</b> RenderX Community Support List<br><b>Subject:</b> [xep-support] OutOfMemory issues with XEP4.19<o:p></o:p></span></p></div><p class=MsoNormal><o:p> </o:p></p><div><p class=MsoNormal><span class=apple-style-span>Hi,</span><o:p></o:p></p></div><div><p class=MsoNormal><span class=apple-style-span>We recently discovered some memory issues with XEP4.19 during load testing.</span><o:p></o:p></p></div><div><p class=MsoNormal><span class=apple-style-span>The test comprised of multiple (about 5000 total renders) concurrent (no more than 5) FO to PDF renders of the same two page input.</span><o:p></o:p></p></div><div><p class=MsoNormal><span class=apple-style-span>The input contains both tables and images.</span><o:p></o:p></p></div><div><p class=MsoNormal><span class=apple-style-span>We found that the memory usage slowly crept up over time a heap dump revealed the following:</span><o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><p class=MsoNormal><span class=apple-style-span>71,183,440 (28.34%) [32] 5 class com/speedlegal/genericexport/XEPExport 0xdd55ba10 </span><o:p></o:p></p><div><p class=MsoNormal><span class=apple-style-span>71,183,408 (28.34%) [16] 1 com/renderx/xep/FormatterImpl 0xde359840 </span><o:p></o:p></p></div><div><p class=MsoNormal><span class=apple-style-span>71,183,392 (28.34%) [16] 1 com/renderx/xep/FormatterCore 0xdba87a10 </span><o:p></o:p></p></div><div><p class=MsoNormal><span class=apple-style-span>71,183,376 (28.34%) [336] 23 com/renderx/xep/lib/Conf 0xde375fd8 </span><o:p></o:p></p></div><div><p class=MsoNormal><span class=apple-style-span>67,949,056 (27.05%) [72] 10 com/renderx/xep/cmp/ImageFactory 0xdb32e790 </span><o:p></o:p></p></div><div><p class=MsoNormal><span class=apple-style-span>65,786,552 (26.19%) [32] 1 com/renderx/util/Hashtable 0xdb32e9b8</span><o:p></o:p></p><p class=MsoNormal><span style='font-family:"Arial","sans-serif";color:blue'>65,786,520 (26.19%) [1,048,592] 91,000 array of java/lang/Object 0xe2864560</span><o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal><span style='font-family:"Arial","sans-serif";color:blue'>This basically means that our FormatterImpl object was holding into image references which could not be garbage collected. We were using a single FormatterImpl instance across threads to avoid the overhead of creating this object over and over. We did not and could not call the cleanup() method during this testing as we understand this cannot be called during rendering.</span><o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal><span style='font-family:"Arial","sans-serif";color:blue'>We made sure that TMPDIR and the appropriate MEMOIZE settings were in place but this made no difference.</span><o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal><span style='font-family:"Arial","sans-serif";color:blue'>Using a new instance of the FormatterImpl for each render resolved the memory issues.</span><o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal><span style='font-family:"Arial","sans-serif";color:blue'>Questions:</span><o:p></o:p></p><p class=MsoNormal><span style='font-family:"Arial","sans-serif";color:blue'>- is this a known issue, bug or limitation? </span><o:p></o:p></p><p class=MsoNormal><span style='font-family:"Arial","sans-serif";color:blue'>- is correct use of the FormatterImpl documented somewhere? The forums suggest that FormatterImpl should be used this way but cleanup should be called from time to time. Perhaps cleanup() could block if a render is in progress </span><o:p></o:p></p></div><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal><span style='font-family:"Arial","sans-serif";color:blue'>Regards,</span><o:p></o:p></p><p class=MsoNormal><span style='font-family:"Arial","sans-serif";color:blue'>Justin,</span><o:p></o:p></p><p class=MsoNormal><br><br>-- <o:p></o:p></p><div><div><div><p><span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>Regards,<o:p></o:p></span></p><p><b><span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>Justin Lipton</span></b><span style='font-size:10.0pt;font-family:"Arial","sans-serif"'><o:p></o:p></span></p><div><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Arial","sans-serif";color:#666666'><img width=96 height=34 id="_x0000_i1025" src="http://exari.com/Exari/media/Exari-Resources/Email%20Signature/exari-logo.png"></span><span style='font-size:10.0pt;font-family:"Arial","sans-serif"'><o:p></o:p></span></p></div><div><p><span style='font-size:10.0pt;font-family:"Arial","sans-serif";color:#666666'>Email: <a href="mailto:justin.lipton@exari.com" target="_blank">justin.lipton@exari.com</a><br>Office: +613 9621 2775<br>Mobile: +614 0958 9943</span><span style='font-size:10.0pt;font-family:"Arial","sans-serif"'><o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Arial","sans-serif"'><a href="http://www.exari.com/demo-trial.html" target="_blank">Demo</a> <a href="http://blog.exari.com/" target="_blank">Blog</a> <a href="http://www.twitter.com/exari" target="_blank">Twitter</a> <a href="http://www.linkedin.com/companies/exari-systems" target="_blank">LinkedIn</a><o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Arial","sans-serif"'> <o:p></o:p></span></p></div><p class=MsoNormal style='margin-bottom:12.0pt'><span style='font-size:10.0pt;font-family:"Arial","sans-serif";color:#666666'>Boston | London | Melbourne | Munich</span><span style='font-size:10.0pt;font-family:"Arial","sans-serif"'><o:p></o:p></span></p><div><p class=MsoNormal><span style='font-size:7.0pt;font-family:"Arial","sans-serif";color:#7F7F7F'>This e-mail message is transmitted to you by Exari Group, Inc.. This e-mail contains information which may be confidential and legally privileged, and is for the exclusive use of the intended recipient. If you are not the intended recipient, any disclosure, copying, distribution or use of this information is prohibited. If you have received this e-mail in error, please notify us immediately by telephone +1 617.938.3777 or by e-mail and then delete the message from your computer.</span><span style='font-size:10.0pt;font-family:"Arial","sans-serif"'><o:p></o:p></span></p></div></div></div></div><p class=MsoNormal><br>!DSPAM:87,503bd15263731548410865! <o:p></o:p></p></div></body></html>