$ontext ===============================================================

LAST MODIFIED: 2008/11/08

FILENAME: pprocess.gms

PURPOSE:  This tool performs all performance post-processing for a
          given set of trace files. For all paris of trace files,
          the routine runs the following performance tool routines:

          -solver square
          -resource time comparison
          -performance profiles
          -performance profiles (absolute time)

REFERENCE:

          Performance World Website, Ptools section, online at
          http://www.gamsworld.eu/performance/tools.htm.


REQUIRED INPUTS:
	  --trace1:    trace file of solver 1
	  --trace2:    trace file of solver 2 etc up to --trace8

OPTIONAL INPUTS:
          --outdir:    output directory (default: results/)
          --outfile:   output filename (default: results.htm)
          --useobjest: flag to use the objective value estimate in
                       the square.gms and inconsistencies.gms routines.
                       Also generates performance profile plots.
          --modelfile  modelfile containing models to be considered
                       in comparison
          --modellib   the model library where the models are taken from.
                       Valid values are linlib, globallib, minlplib, none.
                       (default: none)

OPTIONAL PROFILE INPUTS:
          --allsolver: if set, adds cumulative solver curves (ALL_SEQ,
                       ALL_PAR, ALL_MIN) to absolute time profiles
          --factor     factor for values of tau: 1, factor, factor^2,
                       factor^3 (default 1.25)
          --numpts:    number of data points for each solver for
                       performance profile (default: 30)
          --delta      actual relative tolerance for having exact best
                       objective value. Because of rounding errors,
                       this should generally not be exactly zero.
                       (default: 1e-5)
          --gaptol     actual relative gap tolerance for having solved
                       a model. Because of rounding errors,
                       this should generally not be exactly zero.
                       (default: 1e-6)
	  --resmin:    minimum resource time threshold. If a solver
                       reports a resource time below --resmin, then
                       it is increased to resmin before being used
                       in ratios. (default 0.05).
          --resmax     maximum resource time for absolute time performance
                       profiles

OPTIONAL SQUARE INPUTS:
          --bnd:       relative objective function difference. Prints
                       "optimal" solution in boldface for HTML output
                       for one solver if the relative objective function
                       difference is greater than --bnd. (default 1e-5)
          --resmin:    minimum resource time threshold. If both solvers
                       report a resource time below --resmin, then no
                       ratio is reported. In this case, both solver times
                       are considered the same (default 0.05).

OPTIONAL RESTIME INPUTS:
          --bnd:     real. Bound for optimal outcome. Outcome is
                     optimal if both solvers
                     scaled solutions are within bnd of eachother,
                     i.e. relative error (default: 1e-5)
          --resmin:  minimum resource time threshold. If both solvers
                     report a resource time below --resmin, then no
                     ratio is reported. In this case, both solver
                     times are considered the same (default 0.05).
          --tsame:   bound in % for solver res. times to be considered
                     same (default 10%)
          --tfaster: bound in % for solver res. times to be considered
                     much faster (default 50%)

OPTIONAL TIMINGS INPUTS:
          --colselect: selects a tracefile column to be additionally printed
                       to resource time (default: ETSolver)
          --meanonopt: whether the mean time w.r.t. all models solved to
                       optimality should be computed (default: 0)
          --reslim:    limit used for resource value. Unsolved models are
                       counted with twice the reslim in the calculation
                       of mean values (default: max resource usage among all data).

REMARKS:
	  User must input one tracefiles for each solver. Currently
          works for LP, RMIP, NLP, RMINLP, MIP, and MINLP  type models.

	  Each trace file can have model/solve information for only 1
          solver. For example trace1.trc can have info for all LPs in
          PerformanceLib for solver A, and trace2.trc can have
          information for all LPs using solver B.

          Tracefiles containing multiple solvers will cause error
          messages upon compilation.

$offtext ==============================================================

*=== Error checks
$if not set trace1     $goto errors_notrace1
$if not set trace2     $goto errors_notrace2
$if not exist %trace1% $goto errors_trace1notexist
$if not exist %trace2% $goto errors_trace2notexist


*=== Check if user specified all options in an --optfile
$if set optfile $include %optfile%

$if not set useobjest $set useobjest 0
$if not set uselog    $set uselog    0

*=== Set default pprocess.gms options
$if not set delta     $set delta 1e-5
$if not set resmax    $set resmax 1000
$if not set gaptol    $set gaptol 1e-5
$if not set outfile   $set outfile results
$if not set outdir    $set outdir  results
$if not set useobjest $set useobjest 0

$if     set allsolver $set allsolver --allsolver=yes
$if     set allsolver $goto allsolverset
$if not set allsolver $set allsolver
$label allsolverset

*=== Set default square.gms options
$if not set bnd     $set bnd     1e-5
$if not set resmin  $set resmin  0.05


*=== Set default restime.gms options
$if not set tsame   $set tsame   10
$if not set tfaster $set tfaster 50


$ set  sep /


*=== Indicator if PAVER run
$if not set paver  $set paver    0
$if not set numpts $set numpts  30


*=== Determine trace data files that are input
$echo > tracedata
$echo > alltracedata

$if set trace1 $echo '"%trace1%"' >> tracedata
$if set trace1 $echo '"%trace1%"' >> alltracedata
$if set trace1 $set trace1def 1
$if not set trace1 $set trace1def 0

$if set trace2 $echo '"%trace2%"' >> tracedata
$if set trace2 $echo '"%trace2%"' >> alltracedata
$if set trace2 $set trace2def 1
$if not set trace2 $set trace2def 0

$if set trace3 $echo '"%trace3%"' >> tracedata
$if set trace3 $echo '"%trace3%"' >> alltracedata
$if set trace3 $set trace3def 1
$if not set trace3 $set trace3def 0

$if set trace4 $echo '"%trace4%"' >> tracedata
$if set trace4 $echo '"%trace4%"' >> alltracedata
$if set trace4 $set trace4def 1
$if not set trace4 $set trace4def 0

$if set trace5 $echo '"%trace5%"' >> tracedata
$if set trace5 $echo '"%trace5%"' >> alltracedata
$if set trace5 $set trace5def 1
$if not set trace5 $set trace5def 0

$if set trace6 $echo '"%trace6%"' >> tracedata
$if set trace6 $echo '"%trace6%"' >> alltracedata
$if set trace6 $set trace6def 1
$if not set trace6 $set trace6def 0

$if set trace7 $echo '"%trace7%"' >> tracedata
$if set trace7 $echo '"%trace7%"' >> alltracedata
$if set trace7 $set trace7def 1
$if not set trace7 $set trace7def 0

$if set trace8 $echo '"%trace8%"' >> tracedata
$if set trace8 $echo '"%trace8%"' >> alltracedata
$if set trace8 $set trace8def 1
$if not set trace8 $set trace8def 0


*=== Create set of trace files
$set tracefile tracedata
Set tfiles "tracefiles"  /
$onempty
$include %tracefile%
$offempty
/;
Alias (tfilesA, tfilesB, tfiles);


*=== Make output directory and copy trace and other files
execute 'mkdir %outdir%'
execute 'cp style.css %outdir%'

if(%trace1def%, execute 'cp -f %trace1% %outdir%' );
if(%trace2def%, execute 'cp -f %trace2% %outdir%' );
if(%trace3def%, execute 'cp -f %trace3% %outdir%' );
if(%trace4def%, execute 'cp -f %trace4% %outdir%' );
if(%trace5def%, execute 'cp -f %trace5% %outdir%' );
if(%trace6def%, execute 'cp -f %trace6% %outdir%' );
if(%trace7def%, execute 'cp -f %trace7% %outdir%' );
if(%trace8def%, execute 'cp -f %trace8% %outdir%' );


*==========================================================
*=== Run performance tools
*==========================================================

file ftmp / pprocessbat.gms /;
ftmp.pw = 10000;

*=== Parmfile with all trace files
put ftmp   '*'
         / '* Batch file created on %system.date% %system.time% '
         / '*'
       /// '* Create List of Trace input files' ;

loop(tfiles,
   put ftmp / '$call echo  --trace' ord(tfiles):0:0 '=' 
                  tfiles.tl:0 ' >> tfiles.inc'
);

*=== Run solver square, resource time, and performance ====
*=== profile utilities ====================================

put ftmp // '*=== Batch runs for solver square utility' ;
loop( (tfilesA, tfilesB)$(not sameas(tfilesA,tfilesB) and
                                ord(tfilesB)>ord(tfilesA) ),

   put ftmp / '$call =gams square.gms ';
            if(tfiles(tfilesA),
               put ' --trace1=' tfilesA.tl:0;
            );
            if(tfiles(tfilesB),
               put ' --trace2=' tfilesB.tl:0;
            );
   put      ' --outfile=' tfilesA.tl:0 '_' tfilesB.tl:0
            ' --bnd=%bnd% --resmin=%resmin%'
            ' --outdir=%outdir% --useobjest=%useobjest%' 
            ' --gaptol=%gaptol% --uselog=%uselog%';
$if set reslim
   put      ' --reslim=%reslim%'
$if set modellib
   put      ' --modellib=%modellib%'
$if set modelfile
   put      ' --modelfile=%modelfile%'
   put      /;

   put ftmp / '$if errorlevel 1 $abort "Problems with square.gms"' /;
);



put ftmp // '*=== Batch runs for resource time utility';
loop( (tfilesA, tfilesB)$(not sameas(tfilesA,tfilesB) and
                                ord(tfilesB)>ord(tfilesA) ),

   put ftmp / '$call =gams restime.gms ';
            if(tfiles(tfilesA),
               put ' --trace1=' tfilesA.tl:0;
            );
            if(tfiles(tfilesB),
               put ' --trace2=' tfilesB.tl:0;
            );
   put      ' --outfile=' tfilesA.tl:0 '_' tfilesB.tl:0
            ' --bnd=%bnd% --resmin=%resmin% --tsame=%tsame%'
            ' --tfaster=%tfaster%'
            ' --outdir=%outdir%';
$if set modelfile
   put      ' --modelfile=%modelfile%'
   put      /;

   put ftmp / '$if errorlevel 1 $abort "Problems with restime.gms"' /;
);
put ftmp //;


set objwithin/0 "0",Inf "Inf"/;

put ftmp // '*=== Batch runs for performance profile utility';
   loop(objwithin,
      put ftmp / '$call =gams plotprof.gms parmfile="tfiles.inc" ';
      put ftmp ' --otype=png --outdir=%outdir% --figname=profile_' objwithin.te(objwithin):0 ;
      if( sameas(objwithin,'0'),
           put ' --delta= %delta% '
      else
           put ' --delta=' objwithin.tl:0 ;
      );
$if set colselect
      put ' --colselect=%colselect% ';
      put      ' --numpts=%numpts% --outfile=profile_' objwithin.te(objwithin):0 '.txt '
*              ' --factor=%factor% --resmin=%resmin% '
$if set modelfile
      put ' --modelfile=%modelfile% '
      put /;
      put ftmp '$call =gams plotprof.gms parmfile="tfiles.inc" ';
      put ftmp ' --otype=ps --outdir=%outdir% --figname=profile_' objwithin.te(objwithin):0 ;
      if( sameas(objwithin,'0'),
           put ' --delta= %delta% '
      else
           put ' --delta=' objwithin.tl:0 ;
      );
$if set colselect
      put ' --colselect=%colselect% ';
      put      ' --numpts=%numpts% --outfile=profile_' objwithin.te(objwithin):0 '.txt '
*              ' --factor=%factor% --resmin=%resmin% '
$if set modelfile
      put ' --modelfile=%modelfile% '
      put /;
      put ftmp / '$if errorlevel 1 $abort "Problems with plotprof.gms"' /;
   );

put ftmp // '*=== Batch runs for performance profile (absolute time) utility';
   loop(objwithin,
      put ftmp / '$call =gams plotproft.gms parmfile="tfiles.inc" ';
      put ftmp ' --otype=png --outdir=%outdir% --figname=profilet_' objwithin.te(objwithin):0 ;
      if( sameas(objwithin,'0'),
           put ' --delta= %delta% '
      else
           put ' --delta=' objwithin.tl:0 ;
      );
$if set colselect
      put ' --colselect=%colselect% ';
      put      ' --numpts=%numpts% --outfile=profilet_' objwithin.te(objwithin):0 '.txt '
               ' --resmax=%resmax% %allsolver% '
$if set modelfile
      put ' --modelfile=%modelfile% '
      put /;
      put ftmp '$call =gams plotproft.gms parmfile="tfiles.inc" ';
      put ftmp ' --otype=ps --outdir=%outdir% --figname=profilet_' objwithin.te(objwithin):0 ;
      if( sameas(objwithin,'0'),
           put ' --delta= %delta% '
      else
           put ' --delta=' objwithin.tl:0 ;
      );
$if set colselect
      put ' --colselect=%colselect% ';
      put      ' --numpts=%numpts% --outfile=profilet_' objwithin.te(objwithin):0 '.txt '
               ' --resmax=%resmax% %allsolver% '
$if set modelfile
      put ' --modelfile=%modelfile% '
      put /;

      put ftmp / '$if errorlevel 1 $abort "Problems with plotproft.gms"' /;
   );


set gapwithin/0 "0", "0.1" "10", Inf "Inf"/
set objgapwithin(objwithin, gapwithin) / 0.0, 0.'0.1', 'Inf'.'0.1' /;
if(%useobjest%>0,
   put ftmp // '*=== Batch runs for performance profile (use obj estimate) '
               'utility' ;
   loop((objwithin,gapwithin)$objgapwithin(objwithin,gapwithin),
 
      put ftmp '$call =gams plotprofg.gms parmfile="tfiles.inc" --outdir=%outdir% ';
      put ftmp ' --otype=png --figname=profileg_' objwithin.te(objwithin):0 '_' gapwithin.te(gapwithin):0; 
      if( sameas(objwithin,'0'),
           put ' --delta= %delta% ' ;
      else
           put ' --delta=' objwithin.tl:0 ;
      );
      if ( sameas(gapwithin,'0'),
           put ' --gaptol=%gaptol% '
      else
           put ' --gaptol=' gapwithin.tl:0;
     );
$if set colselect
      put ' --colselect=%colselect% ';
      put ' --numpts=%numpts% '
          ' --outfile=profileg_' objwithin.te(objwithin):0 '_' gapwithin.te(gapwithin):0 '.txt '
$if set modelfile
      put ' --modelfile=%modelfile% '
      put /;

      put ftmp '$call =gams plotprofg.gms parmfile="tfiles.inc" --outdir=%outdir% ';
      put ftmp ' --otype=postscript --figname=profileg_' objwithin.te(objwithin):0 '_' gapwithin.te(gapwithin):0; 
      if( sameas(objwithin,'0'),
           put ' --delta= %delta% '
      else
           put ' --delta=' objwithin.tl:0 ;
      );
      if ( sameas(gapwithin,'0'),
           put ' --gaptol=%gaptol% '
      else
           put ' --gaptol=' gapwithin.tl:0;
      );
$if set colselect
      put ' --colselect=%colselect% ';
      put ' --numpts=%numpts% '
          ' --outfile=profileg_' objwithin.te(objwithin):0 '_' gapwithin.te(gapwithin):0 '.txt '
$if set modelfile
      put ' --modelfile=%modelfile% '
      put /;
   );

   put ftmp / '$if errorlevel 1 $abort "Problems with plotprofg.gms"' /;
);


put ftmp // '*=== Model statistics and timings' ;
put ftmp / '$call =gams timings.gms --outdir=%outdir% '
           '--useobjest=%useobjest% parmfile=tfiles.inc ';
   
$if set reslim
   put ' --reslim=%reslim%'
$if set modellib
   put ' --modellib=%modellib%'
$if set colselect
   put ' --colselect=%colselect%'
$if set meanonopt
   put ' --meanonopt=%meanonopt%'
$if set modelfile
   put ' --modelfile=%modelfile%'
put /;
put ftmp / '$if errorlevel 1 $abort "Problems with timings.gms"' /;


* Perform data inconsistencies check
put ftmp // '*=== Data inconsistency check' ;
put ftmp / '$call =gams inconsistencies.gms ';
loop(tfiles,
   put ' --trace' ord(tfiles):0:0 '=' tfiles.tl:0;
);
put ' --useobjest=%useobjest% --outdir=%outdir% '
$if set modelfile
  put ' --modelfile=%modelfile% '
put ftmp / '$if errorlevel 1 $abort "Problems with inconsistencies.gms"' /;

putclose ftmp // "$call 'rm -f tfiles.inc'";

execute '=gams pprocessbat.gms';
scalar rc;
rc=errorlevel;
abort$rc 'problems with pprocessbat.gms';
execute 'mv -f pprocessbat.gms %outdir%%sep%'


*==========================================================
*=== Create main performance summary HTM file
*==========================================================

file fhtm / "%outdir%/%outfile%.htm" /;
put fhtm;

put / '<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">'
    / '<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">'
    / '<head>'
    / '   <link rel="stylesheet" type="text/css" href="style.css" media="screen,projection" />'
    / '   <title>Performance Results</title>'
    / '</head>'
   // '<body>'
   // '<div id="container">'
   // '<div id="main">'

    / '<h2>Performance Results</h2>' /;


put / '<p class="block">Results created on %system.date% %system.time% '
    / 'using the PAVER Server/Performance Tools available at'
    / '<a href="http://www.gamsworld.eu/performance/paver">'
    / 'http://www.gamsworld.eu/performance/paver</a><br>'
    / '<br>This page is a summary of the performance tests, using the'
    / 'solver square, resource time, and performance profile utilities.'
    / 'The Performance Tools  are described in detail in the Performance'
    / 'World webpage at <a href="http://www.gamsworld.eu/performance">'
    / 'http://www.gamsworld.eu/performance</a>.'
    / 'For a list of model and solve status codes see the '
    / '<a href="status_codes.htm">Status Codes Section</a>.'
    / '</p>';

put / '<h3>Trace files used:</h3>';

put / '<table>'
    / '<tr>'
    / '   <th>Input File Number</th>'
    / '   <th>Filename</th>'
    / '</tr>';

loop(tfiles,

   put / '<tr>'
       / '   <td align=left><b>Tracefile ' ord(tfiles):0:0  ':</b></td>'
       / '   <td align=right><a href="' tfiles.tl:0 '">' tfiles.tl:0 '</a></td>'
       / '</tr>';

);
put / '</table><br>'/;



*=== Performance profile summary ==========================
put / '<hr>'

put / '<h3>Performance profile summary:</h3>'
put / '<p class="block">The profile utility compares solver outcomes of'
    / 'any number of given solvers (up to 8) over a model test set. The '
    / 'tool is described in detail at '
    / '<a href="http://www.gamsworld.eu/performance/pprofile.htm">'
    / 'http://www.gamsworld.eu/performance/pprofile.htm</a>.</p>'

put / '<ul>'
    / '    <li>Performance profiles for all objective value tolerances: '
      '<a href="profile_summary.htm">profile_summary.htm</a> (compares '
      'objective values)';
$ifthen %useobjest% == 1
put / '    <li>Performance profiles for all objective value tolerances '
      'and gap tolerances: '
      '<a href="profileg_summary.htm">profileg_summary.htm</a> (compares '
      'objective values and gap tolerances)';
$endif
put
    / '</ul>'


*=== Solver square summary ================================
put / '<hr>'

put / '<h3>Solver square summary:</h3>'
    / '<p class="block">The solver square utility compares solver outcomes'
    / 'of two given solvers over a model test set. The tool is described '
    / 'in detail at <a href="http://www.gamsworld.eu/performance/square.htm">'
    / 'http://www.gamsworld.eu/performance/square.htm</a>.</p>'

put / '<table>'
    / '<tr>'
    / '    <th></th>';
loop(tfilesB,
   put / '    <th align=right><b>' tfilesB.tl:0 '</b></th>';
);
put / '</tr>';


loop(tfilesA,
   put / '<tr>'
       / '    <td class="tint"><b>' tfilesA.tl:0 '</b></td>';

   loop(tfilesB,
      if(ord(tfilesB)>ord(tfilesA),
         put / '    <td align=right><a href="' tfilesA.tl:0 '_' tfilesB.tl:0
               '_sqr.htm ">results</a></td>';
      else
         put / '    <td align=right>--</td>';
      );
   );

   put / '</tr>';
);
put / '</table>';



*=== Resource time summary ================================
put '<hr>'

put / '<h3>Resource time summary:</h3>'
    / '<p class="block">The resource time utility compares solver '
    / 'resource times of two given solvers over a model test set. '
    / 'The tool is described in detail at'
    / '<a href="http://www.gamsworld.eu/performance/restime.htm">'
    / 'http://www.gamsworld.eu/performance/restime.htm</a>.</p>'


put / '<table>'
    / '<tr>'
    / '    <th></th>';
loop(tfilesB,
   put / '    <th align=right><b>' tfilesB.tl:0 '</b></th>';
);
put / '</tr>';


loop(tfilesA,
   put / '<tr>'
       / '    <td class="tint"><b>' tfilesA.tl:0 '</b></td>';

   loop(tfilesB,
      if(ord(tfilesB)>ord(tfilesA),
         put / '    <td align=right><a href="' tfilesA.tl:0 '_' tfilesB.tl:0
               '_res.htm ">results</a>'
               '</td>';
      else
         put / '    <td align=right>--</td>';
      );
   );
   put / '</tr>';
);
put / '</table>';

put / '<hr>'
    / '<h3>Timing Information</h3>'
    / '<p class="block">For simple timing information (traceopt=3 '
    / 'files only), see the resource time benchmarking tables:</p>'/
    / '<ul>'
    / '   <li><a href="timings.htm">timings.htm</a></li>'
    / '</ul>'
    / '<p>';

put / '<hr>'
    / '<h3>Data Inconsistency Check</h3>'
    / '<p class="block">Data inconsistencies are checked for differences'
    / 'in wall time vs CPU time and objective estimate versus '
    / 'actual objective value. These checks may not be applicable for '
    / 'all models or all trace files.</p>'
    / '<ul>'
    / '   <li><a href="inconsistencies.htm">inconsistencies.htm</a></li>'
    / '</ul>'
    / '<p>';


put / '</div><!-- end div=main -->'
    / '<div id="footer">'
    / '<p>Generated using the PAVER Server/Performance Tools.</a></p>'
   // '</div>'
   // '</div> <!-- end div=container -->'
   // '</body>'
   // '</html>'
putclose fhtm;


*==========================================================
*=== Create performance profile summary HTM file
*=== profile_summary.htm
*==========================================================
file fhtm2 / "%outdir%/profile_summary.htm" /;
put fhtm2;

put / '<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">'
    / '<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">'
    / '<head>'
    / '   <link rel="stylesheet" type="text/css" href="style.css" media="screen,projection" />'
    / '   <title>Performance Profile Summary</title>'
    / '</head>'
   // '<body>'
   // '<div id="container">'
   // '<div id="main">'

    / '<h2>Performance Profile Summary</h2>' /;

put / '<p class="block">Results created on %system.date% %system.time% '
       / 'using the PAVER Server/Performance tools available at'
       / '<a href="http://www.gamsworld.eu/performance/paver">'
       / 'http://www.gamsworld.eu/performance/paver</a></p>'


put / '<p>Performance profiles for different objective value tolerances.'
    / 'The profile utility compares solver outcomes of any number of given '
    / 'solvers (up to 8) over a model test set. The tool is described in detail'
    / 'at <a href="http://www.gamsworld.eu/performance/pprofile.htm">'
    / 'http://www.gamsworld.eu/performance/pprofile.htm</a>.</p>';

put / '<p>A solver is considered feasible or optimal, if it has the proper model'
    / 'and solver return codes and the <i>relative objective value error</i> is within'
    / '<i>n</i> % of the best possible solution. If the best found objective'
    / 'value is 0, then the <i>absolute objective value error</i> is used.</p>';

put / '<p>The following relative objective value tolerances are used:</p>'
    / '<ul>';
loop(objwithin,
    if( sameas(objwithin,'Inf'),
       put / '    <li>No objective value tolerance, i.e. we do not check if obj is best found (within ' objwithin.te(objwithin) '%).'
    elseif( sameas(objwithin,'0')),
       put / '   <li>Only best objective value (i.e. ' objwithin.te(objwithin):0 ' % of best solution found.'
           / '   <b>Actual tolerance used is %delta%</b> using relative error)';
    else
       put / '   <li>Relative objective value error is within ' objwithin.te(objwithin):0 '% with respect to best value.';
    );
);
put / '</ul>';

put / 'There is a curve for an additional <i>super solver</i> shown:</p>' /
    / '<p>'
    / '<ul>'
    / '   <li>CAN_SOLVE: solver consisting of all solvers. Solver time '
    / '       used is the minimum of all solver times for a given model.'
    / '</ul>'
    / '</p>';



loop(objwithin,

   if( sameas(objwithin,'Inf'),
       put
       /'<table>'
       /'  <tr align=center>'
       /'    <td class="noborder">'
       /'       <b>Efficiency Only (within ' objwithin.te(objwithin):0 '% wrt to best value) </b><br/>'
       /'          Data File: <a href="profile_' objwithin.te(objwithin):0  '.txt"> profile_'
         objwithin.te(objwithin):0 '.txt</a><br/>'
       /'          Black and White EPS file - '
       /'             <a href="profile_' objwithin.te(objwithin):0 '.eps">profile_' objwithin.te(objwithin):0 '.eps</a><br/>'
       /'          <IMG SRC="profile_' objwithin.te(objwithin):0 '.png">'
       /'    </td>'
       /'  </tr>'
       /'</table>';
   elseif( sameas(objwithin,'0')),
       put
       /'<table>'
       /'   <tr align=center>'
       /'     <td class="noborder">'
       /'       <b>Quality of Solution (within ' objwithin.te(objwithin):0 '% wrt to best value) </b><br/>'
       /'          Data File: <a href="profile_' objwithin.te(objwithin):0  '.txt"> profile_'
         objwithin.te(objwithin):0 '.txt</a><br/>'
       /'          Black and White EPS file: '
       /'             <a href="profile_' objwithin.te(objwithin):0 '.eps">profile_' objwithin.te(objwithin):0 '.eps</a><br/>'
       /'          <IMG SRC="profile_' objwithin.te(objwithin):0 '.png">'
       /'    </td>'
       /'  </tr>'
       /'</table>';
   else
       put
       /'<table>'
       /'  <tr align=center>'
       /'    <td class="noborder">'
       /'      <b>(Within ' objwithin.te(objwithin):0 '% wrt to best value) </b><br/>'
       /'         Data File: <a href="profile_' objwithin.te(objwithin):0  '.txt"> profile_'
         objwithin.te(objwithin):0 '.txt</a><br/>'
       /'         Black and White EPS file: '
       /'            <a href="profile_' objwithin.te(objwithin):0 '.eps">profile_' objwithin.te(objwithin):0 '.eps</a><br/>'
       /'         <IMG SRC="profile_' objwithin.te(objwithin):0 '.png">'
       /'    </td>'
       /'  </tr>'
       /'</table>'
   );
);



*== Add absolute time profile plots
put / '<h3>Absolute Time Performance Profile Summary</h3>'
    / '<p>Below are <i>absolute time performance profile plots</i>. In this case, the '
    / 'performance metric uses absolute time instead of the fastest solver time. That is '
    / 'a model/solver pair passes if it solves the problem within the absolute time given. '
    / 'Objective value tolerances used are the same as for the regular performance profile '
    / 'plots as above.'
$if not set allsolver $goto skipallsolverhtml
    / 'There are curves for three additional <i>super solvers</i> shown: '
    / '<ul>'
    / '   <li>ALL_SEQ: solver consisting of all solvers run in sequence. Solver time '
    / '       used is the sum of all solver times.'
    / '   <li>ALL_PAR: solver consisting of all solvers run in parallel. Solver time '
    / '       used is the max over all solver times.'
    / '</ul>' ;
$label skipallsolverhtml
put / '</p>';

loop(objwithin,

   if( sameas(objwithin,'Inf'),
       put
       /'<table>'
       /'  <tr align=center>'
       /'    <td class="noborder">'
       /'       <b>Efficiency Only (within ' objwithin.te(objwithin):0 '% wrt to best value) </b><br/>'
       /'          Data File: <a href="profilet_' objwithin.te(objwithin):0  '.txt"> profilet_'
         objwithin.te(objwithin):0 '.txt</a><br/>'
       /'          Black and White EPS file - '
       /'             <a href="profilet_' objwithin.te(objwithin):0 '.eps">profilet_' objwithin.te(objwithin):0 '.eps</a><br/>'
       /'          <IMG SRC="profilet_' objwithin.te(objwithin):0 '.png">'
       /'    </td>'
       /'  </tr>'
       /'</table>';
   elseif( sameas(objwithin,'0')),
       put
       /'<table>'
       /'   <tr align=center>'
       /'     <td class="noborder">'
       /'       <b>Quality of Solution (within ' objwithin.te(objwithin):0 '% wrt to best value) </b><br/>'
       /'          Data File: <a href="profilet_' objwithin.te(objwithin):0  '.txt"> profilet_'
         objwithin.te(objwithin):0 '.txt</a><br/>'
       /'          Black and White EPS file: '
       /'             <a href="profilet_' objwithin.te(objwithin):0 '.eps">profilet_' objwithin.te(objwithin):0 '.eps</a><br/>'
       /'          <IMG SRC="profilet_' objwithin.te(objwithin):0 '.png">'
       /'    </td>'
       /'  </tr>'
       /'</table>';
   else
       put
       /'<table>'
       /'  <tr align=center>'
       /'    <td class="noborder">'
       /'      <b>(Within ' objwithin.te(objwithin):0 '% wrt to best value) </b><br/>'
       /'         Data File: <a href="profilet_' objwithin.te(objwithin):0  '.txt"> profilet_'
         objwithin.te(objwithin):0 '.txt</a><br/>'
       /'         Black and White EPS file: '
       /'            <a href="profilet_' objwithin.te(objwithin):0 '.eps">profilet_' objwithin.te(objwithin):0 '.eps</a><br/>'
       /'         <IMG SRC="profilet_' objwithin.te(objwithin):0 '.png">'
       /'    </td>'
       /'  </tr>'
       /'</table>'
   );
);


put / '</div>'
    / '<div id="footer">'
    / '<p>Generated using the PAVER Server/Performance Tools.</a></p>'
   // '</div> <!-- end div=main -->'
   // '</div> <!-- end div=container -->'
   // '</body>'
   // '</html>'
putclose fhtm2;



*==========================================================
*=== Create performance profile summary (gaptol) HTM file
*=== profileg_summary.htm
*==========================================================
$if '%objest%' != '1' $goto no_profile_objest

file fhtm3 / "%outdir%/profileg_summary.htm" /;
put fhtm3;

put / '<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">'
    / '<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">'
    / '<head>'
    / '   <link rel="stylesheet" type="text/css" href="style.css" media="screen,projection" />'
    / '   <title>Performance Profile Summary</title>'
    / '</head>'
   // '<body>'
   // '<div id="container">'
   // '<div id="main">'

    / '<h2>Performance Profile Summary (Objective Gap)</h2>' /;

put / '<p class="block">Results created on %system.date% %system.time% '
       / 'using the PAVER Server/Performance tools available at'
       / '<a href="http://www.gamsworld.eu/performance/paver">'
       / 'http://www.gamsworld.eu/performance/paver</a></p>'


put / '<p>Performance profiles for different objective value tolerances.'
    / 'The profile utility compares solver outcomes of any number of given '
    / 'solvers (up to 8) over a model test set. The tool is described in detail'
    / 'at <a href="http://www.gamsworld.eu/performance/pprofile.htm">'
    / 'http://www.gamsworld.eu/performance/pprofile.htm</a>.</p>';

put / '<p>A solver is considered feasible or optimal, if it has the proper model'
    / 'and solver return codes and the <i>relative objective value error</i> is within'
    / '<i>n</i> % of the best possible solution. If the best found objective'
    / 'value is 0, then the <i>absolute objective value error</i> is used.</p>';

put / '<p>'
    / 'The following resource is used as measurement: ';
$if not set colselect
put / '<b>from solver reported time</b>'
$if set colselect
put / '<b>%colselect%</b>'

put / '</p><p>'
    / 'The following relative objective value tolerances are used:'
    / '<ul>';
loop(objwithin,
    if( sameas(objwithin,'Inf'),
       put / '    <li>No objective value tolerance, i.e. we do not check if obj '
           / 'is best found (within ' objwithin.te(objwithin) '%).'
    elseif( sameas(objwithin,'0')),
       put / '   <li>Only best objective value (i.e. ' objwithin.te(objwithin):0 ' %' 
           / 'of best solution found. <b>Actual tolerance used is %delta%</b> using relative error)';
    else
       put / '   <li>Relative objective value error is within ' objwithin.te(objwithin):0 '%'
           / ' with respect to best value.';
    );
);
put / '</ul></p>';

$ifthen not %useobjest% == 0
put / '<p>The following gap tolerances are used:'
    / '<ul>';
loop(gapwithin,
    if( sameas(gapwithin,'Inf'),
       put / '    <li>No gap tolerance, i.e. we do not check if the gap is closed to a'
           / 'certain value (within ' gapwithin.te(gapwithin) '%).'
    elseif( sameas(gapwithin,'0')),
       put / '   <li>Only closed gap (<b>Actual tolerance used is %gaptol%</b> '
           / 'using relative distance)';
    else
       put / '   <li>Relative gap is at most ' gapwithin.te(gapwithin):0 '%.';
    );
);
put / '</ul></p>';
$endif

loop((objwithin,gapwithin)$objgapwithin(objwithin,gapwithin),
   put /'<table>'
       /'  <tr align=center>'
       /'    <td class="noborder">';
   if( sameas(objwithin,'Inf') and sameas(gapwithin,'Inf'),
       put
       /'       <b>Efficiency Only (within ' objwithin.te(objwithin):0 '% wrt to best value) </b>';
$if not %useobjest% == 0
       put ', gap at most ' gapwithin.te(gapwithin):0 '%)</b>';
   elseif( sameas(objwithin,'0') or not sameas(gapwithin,'Inf')),
       put 
       /'      <br/>Quality of Solution (within ' objwithin.te(objwithin):0 '% wrt to best value';
$if not %useobjest% == 0
       put ', gap at most ' gapwithin.te(gapwithin):0 '%) </b>';
   else
       put
       / '      <br/><b>(Within ' objwithin.te(objwithin):0 '% wrt to best value';
$if not %useobjest% == 0
       put ', gap at most ' gapwithin.te(gapwithin):0 '%) </b></td></tr>';
   );
   put
       /'          <br/>Data File: <a href="profileg_' objwithin.te(objwithin):0 '_' gapwithin.te(gapwithin):0 '.txt"> profileg_'
         objwithin.te(objwithin):0 '_' gapwithin.te(gapwithin):0 '.txt</a><br/>'
       /'          Black and White EPS file - '
       /'             <a href="profileg_' objwithin.te(objwithin):0 '.eps">profileg_' 
           objwithin.te(objwithin):0 '_' gapwithin.te(gapwithin):0 '.eps</a><br/>'
       /'          <IMG SRC="profileg_' objwithin.te(objwithin):0 '_' gapwithin.te(gapwithin):0 '.png">'
       /'    </td>'
       /'  </tr>'
       /'</table>';
);

put / '</div>'
    / '<div id="footer">'
    / '<p>Generated using the PAVER Server/Performance Tools.</a></p>'
   // '</div> <!-- end div=main -->'
   // '</div> <!-- end div=container -->'
   // '</body>'
   // '</html>'
putclose fhtm3;
$label no_profile_objest


*===========================================================
*=== Create HTM file with GAMS model and solver status codes
*===========================================================
file fhtm4 / "%outdir%/status_codes.htm" /;
put fhtm4;

put / '<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">'
    / '<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">'
    / '<head>'
    / '   <link rel="stylesheet" type="text/css" href="style.css" media="screen,projection" />'
    / '   <title>GAMS Model and Solver Status Codes</title>'
    / '</head>'
   // '<body>'
   // '<div id="container">'
   // '<div id="main">'

    / '<h2>GAMS Model and Solver Status Codes</h2>' /;

put / '<p class="block">Below is the list of GAMS model and solver status codes as'
       / 'used in the PAVER Server/Performance tools, available at'
       / '<a href="http://www.gamsworld.eu/performance/paver">'
       / 'http://www.gamsworld.eu/performance/paver</a></p>';

put / '<table>'
    / '<tr>'
    / '   <th><b>MODEL STATUS CODE</b></th>'
    / '   <th><b>DESCRIPTION</b></th>'
    / '</tr>'
    / '<tr>'
    / '   <td><tt>1</tt></td>'
    / '   <td><tt>Optimal</tt></td>'
    / '</tr>'
    / '<tr>'
    / '   <td><tt>2</tt></td>'
    / '   <td><tt>Locally Optimal</tt></td>'
    / '</tr>'
    / '<tr>'
    / '   <td><tt>3</tt></td>'
    / '   <td><tt>Unbounded</tt></td>'
    / '</tr>'
    / '<tr>'
    / '   <td><tt>4</tt></td>'
    / '   <td><tt>Infeasible</tt></td>'
    / '</tr>'
    / '<tr>'
    / '   <td><tt>5</tt></td>'
    / '   <td><tt>Locally Infeasible</tt></td>'
    / '</tr>'
    / '<tr>'
    / '   <td><tt>6</tt></td>'
    / '   <td><tt>Intermediate Infeasible</tt></td>'
    / '</tr>'
    / '<tr>'
    / '   <td><tt>7</tt></td>'
    / '   <td><tt>Intermediate Nonoptimal</tt></td>'
    / '</tr>'
    / '<tr>'
    / '   <td><tt>8</tt></td>'
    / '   <td><tt>Integer Solution</tt></td>'
    / '</tr>'
    / '<tr>'
    / '   <td><tt>9</tt></td>'
    / '   <td><tt>Intermediate Non-Integer</tt></td>'
    / '</tr>'
    / '<tr>'
    / '   <td><tt>10</tt></td>'
    / '   <td><tt>Integer Infeasible</tt></td>'
    / '</tr>'
    / '<tr>'
    / '   <td><tt>11</tt></td>'
    / '   <td><tt>Licensing Problems - No Solution</tt></td>'
    / '</tr>'
    / '<tr>'
    / '   <td><tt>12</tt></td>'
    / '   <td><tt>Error Unknown</tt></td>'
    / '</tr>'
    / '<tr>'
    / '   <td><tt>13</tt></td>'
    / '   <td><tt>Error No Solution</tt></td>'
    / '</tr>'
    / '<tr>'
    / '   <td><tt>14</tt></td>'
    / '   <td><tt>No Solution Returned</tt></td>'
    / '</tr>'
    / '<tr>'
    / '   <td><tt>15</tt></td>'
    / '   <td><tt>Solved Unique</tt></td>'
    / '</tr>'
    / '<tr>'
    / '   <td><tt>16</tt></td>'
    / '   <td><tt>Solved</tt></td>'
    / '</tr>'
    / '<tr>'
    / '   <td><tt>17</tt></td>'
    / '   <td><tt>Solved Singular</tt></td>'
    / '</tr>'
    / '<tr>'
    / '   <td><tt>18</tt></td>'
    / '   <td><tt>Unbounded - No Solution</tt></td>'
    / '</tr>'
    / '<tr>'
    / '   <td><tt>19</tt></td>'
    / '   <td><tt>Infeasible - No Solution</tt></td>'
    / '</tr>'
    / '</table>';

put / '<hr>'

    / '<table>'
    / '<tr>'
    / '   <th><b>SOLVER STATUS CODE</b></th>'
    / '   <th><b>DESCRIPTION</b></th>'
    / '</tr>'
    / '<tr>'
    / '   <td><tt>1</tt></td>'
    / '   <td><tt>Normal Completion</tt></td>'
    / '</tr>'
    / '<tr>'
    / '   <td><tt>2</tt></td>'
    / '   <td><tt>Iteration Interrupt</tt></td>'
    / '</tr>'
    / '<tr>'
    / '   <td><tt>3</tt></td>'
    / '   <td><tt>Resource Interrupt</tt></td>'
    / '</tr>'
    / '<tr>'
    / '   <td><tt>4</tt></td>'
    / '   <td><tt>Terminated by Solver</tt></td>'
    / '</tr>'
    / '<tr>'
    / '   <td><tt>5</tt></td>'
    / '   <td><tt>Evaluation Error Limit</tt></td>'
    / '</tr>'
    / '<tr>'
    / '   <td><tt>6</tt></td>'
    / '   <td><tt>Capability Problems</tt></td>'
    / '</tr>'
    / '<tr>'
    / '   <td><tt>7</tt></td>'
    / '   <td><tt>Licensing Problems</tt></td>'
    / '</tr>'
    / '<tr>'
    / '   <td><tt>8</tt></td>'
    / '   <td><tt>User Interrupt</tt></td>'
    / '</tr>'
    / '<tr>'
    / '   <td><tt>9</tt></td>'
    / '   <td><tt>Error Setup Failure</tt></td>'
    / '</tr>'
    / '<tr>'
    / '   <td><tt>10</tt></td>'
    / '   <td><tt>Error Solver Failure</tt></td>'
    / '</tr>'
    / '<tr>'
    / '   <td><tt>11</tt></td>'
    / '   <td><tt>Error Internal Solver Error</tt></td>'
    / '</tr>'
    / '<tr>'
    / '   <td><tt>12</tt></td>'
    / '   <td><tt>Solve Processing Skipped</tt></td>'
    / '</tr>'
    / '<tr>'
    / '   <td><tt>13</tt></td>'
    / '   <td><tt>Error System Failure</tt></td>'
    / '</tr>'
    / '</table>'

put / '</div>'
    / '<div id="footer">'
    / '<p>Generated using the PAVER Server/Performance Tools.</a></p>'
   // '</div> <!-- end div=main -->'
   // '</div> <!-- end div=container -->'
   // '</body>'
   // '</html>'
putclose fhtm4;


*=== Output message(s) ====================================
file fstat /status.log/;
put fstat   '---'
          / '--- Result summary file is %outfile%.htm, located in'
          / '--- folder %outdir%/.'
          / '---'
putclose fstat;
execute 'head status.log'



$goto noerrors


*=== Error messages
$label errors_notrace1
$log ---   ABORTING --trace1 option required: Trace file not specified..."
$abort "Aborting because --trace1 option not specified"

$label errors_notrace2
$log ---   ABORTING --trace2 option required: Trace file not specified..."
$abort   "Aborting because --trace2 option not specified"

$label errors_trace1notexist
$log "---     ABORTING --trace1 file not found..."
$abort "Aborting because trace file 1 not found"

$label errors_trace2notexist
$log ---     ABORTING --trace2 file not found...
$abort "Aborting because trace file 2 not found"


$label noerrors

execute 'rm -f tracedata alltracedata'