[UPD ES] Filmaffinity 4.0

[Garada]

Me alegro que haya aparecido la solución.
Nadie se iba a imaginar que se había cambiado la aplicación por defecto del archivo VBS

No te olvides de dejar la línea como estaba para que al terminar borre ese archivo temporal.

[MrObama2022]

Ok, so ... vbs works.

For Garada, test this parameter in curl:

Code: Select all

w '%output{' + InstallerPath + curlOutputTmp + '}Download completed with response code %{http_code}\n' 

This is how the code should look now (I DON'T HAVE TESTED IT!)

Code: Select all

  if (fileExists(InstallerPath + curlOutput)) then
    DeleteFile(InstallerPath + curlOutput);  
  curlOutputTmp := curlOutput + '.tmp';
  if (fileExists(InstallerPath + curlOutputTmp)) then
    DeleteFile(InstallerPath + curlOutputTmp);  
    
  sCommand := '-L -w "%output{' + InstallerPath + curlOutputTmp + '}Download completed with response code %{http_code}\n" --output "' + InstallerPath + curlOutput + '" --url "' + address + '" ' + '-H "Accept: text/html, */*" -H "Accept-Language: it" -H "DNT: 1" -H "Priority: u=0, i" -H "Sec-Ch-Ua: \"Not)A;Brand\";v=\"8\", \"Chromium\";v=\"138\", \"Google Chrome\";v=\"138\"" -H "Sec-Ch-Ua-Mobile: ?0" -H "Sec-Ch-Ua-Platform: \"Windows\"" -H "Sec-Fetch-Dest: Document" -H "Sec-Fetch-Mode: Navigate" -H "Sec-Fetch-Site: None" -H "Sec-Fetch-User: ?1" -H "Upgrade-Insecure-Requests: 1" -H "User-Agent: ' + curlUserAgent + '"';

  Sleep(delayBetweenRequest);
  Launch(curlPath, sCommand); 
    
  cnt := 0;
  while ((not FileExists(InstallerPath + curlOutputTmp)) AND (cnt < 200)) Do
  begin
    cnt := cnt + 1;
    Sleep(50);
  end;  
  DeleteFile(InstallerPath + curlOutputTmp);
  if (fileExists(InstallerPath + curlOutput)) then
  begin  
    fileContent := TStringList.Create;
    fileContent.LoadFromFile(InstallerPath + curlOutput);
    [...]

How all this should work: Curl create InstallerPath + curlOutput file and SLOWLY it writes html in the file. Only AFTER the file is closed, Curl create another file, InstallerPath + curlOutputTmp and it writes something inside (Download completed with response code 200). When this file is created you can be 100% sure InstallerPath + curlOutput is closed and ready so you can use it.

Remember: this should work, I tested it but I deleted all the code so maybe you have to fix something with escaping, ', ", % or ... I don't remember now.

I did other test at your curl-only version but I don't like the prompt window (I got the same problem in mine curl-only version, that's why I deleted it). For a curl-only option I think we should wait for a new version of AMC where @antp will add an option to Launch command to hide its window.

[Garada]

Good idea but I'm not sure it's necessary.

CURL should create/open the file for writing and keep it locked until it is closed.
While the file is locked it can't be renamed (MoveFile function), so wait for MoveFile function return success should be a good indicator that the process is complete.

Have you had trimmed results with the HTML file generated by CURL? If so, then all previous paragraph is useless.

But I've made some tests checking the result is complete (looking for the end tag "</html>") and all of them were correct.


And yes, +1 for an udpated version of Launch function with a parameter to hide the program executed.

BTW, on my system, the terminal icon briefly appears in the taskbar but not the entire window on the desktop. I mean, it runs minimized, Is that the case for you?

[Garada]

NOTE 2:

To avoid problems such as PACOIRIPO's, I would change the way the VBS file is executed.

Try this (not tested):

Code: Select all

Launch('wscript.exe', InstallerPath + vbsScript);

[MrObama2022]

Good idea but I'm not sure it's necessary.

CURL should create/open the file for writing and keep it locked until it is closed.
While the file is locked it can't be renamed (MoveFile function), so wait for MoveFile function return success should be a good indicator that the process is complete.

Have you had trimmed results with the HTML file generated by CURL? If so, then all previous paragraph is useless.

I'm not 100% sure curl lock the file (we should double check), that's why I created tmp file. Before the tmp file, my ifs was something like this:

Code: Select all

while ((not FileExists(InstallerPath + curlOutput)) do Sleep(50);
fileContent := TStringList.Create;
fileContent.LoadFromFile(InstallerPath + curlOutput);

and this worked randomly, sometime this worked, sometime I got a blank response (without any pascal/amc error).

Have a look at this or try this powerscript file:

Code: Select all

# USE A HUGE FILE TO DOWNLOAD
# CHANGE X:\C.ts WITH YOUR PATH

$job = Start-Job {
    curl.exe -L 'http://PUT HERE A URL FOR A BIG FILE' --output 'X:\C.ts'
}

# CHANGE X:\C.ts WITH YOUR PATH
while (-not (Test-Path "X:\C.ts")) {
    Start-Sleep -Milliseconds 50
}

# CHANGE X:\C.ts WITH YOUR PATH
if ($job.State -eq 'Running') {
    try {
        $fs = [System.IO.File]::Open("X:\C.ts", 'Open', 'Read', 'ReadWrite')
        Write-Host "File Not Locked."
        $fs.Close()
    }
    catch {
        Write-Host "File Locked."
    }
}

$job | Wait-Job | Remove-Job

But I've made some tests checking the result is complete (looking for the end tag "</html>") and all of them were correct.

It's up to your connection, your pc and how busy is the server. With a slow connection you could have problems and get a blank page.

And yes, +1 for an udpated version of Launch function with a parameter to hide the program executed.

BTW, on my system, the terminal icon briefly appears in the taskbar but not the entire window on the desktop. I mean, it runs minimized, Is that the case for you?

In my first test with your script it worked as you said. But in other tests it starts opening and closing dozens of time and every time I cannot minimize or put another window upon. I don't know why.

[MrObama2022]

NOTE 2:

To avoid problems such as PACOIRIPO's, I would change the way the VBS file is executed.

Try this (not tested):

Code: Select all

Launch('wscript.exe', InstallerPath + vbsScript);

Great!

By the way, I'll be away for two weeks (holydays), so we can chat again at september. Love working with you

[Garada]

Have a great holiday!

When you get back, I will have a script ready with all the ideas that have been contributed.

[Radagast]

First, I would like to thank Antoine and Mickaël for many years of using AMC.

Second, also to thank MrObama2022 for the new scripts, great.

I've made some changes to simplify the code and eliminate the dependency on Powershell and VS.
The following code calls CURL directly and use the pascal script MoveFile to handle the result.
No need to call the function setupScript.

Code: Select all

unit ExternalCurlHandler;

uses StringUtils7552;

const
  tmpDir = ''; // (optional) if you want get your scripts directory clean, set here your tmp working dir, example C:\Users\YOURWINDOWSUSER\AppData\Local\Temp\ . Remember to add \ at the end of the directory
  delayBetweenRequest = 2001;
  curlPath = 'curl.exe'; // if you use Windows 7 or 8 download curl.exe for Windows (it's free) and set here the right path
  curlOutput = 'curlOutput.html';
  curlUserAgent = 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/138.0.0.0 Safari/537.36';

var
  InstallerPath: string;
  
function GetPage5Advanced(address: string; referer: string; cookies: string; content: string; headers: string): string;
var
  cnt: integer;
  fileContent: TStringList;
  curlOutputTmp: string;
  sCommand: string;
begin
  Result := '';

  if (tmpDir <> '') then
    InstallerPath := tmpDir
  else  
    InstallerPath := dirScripts;

  if (fileExists(InstallerPath + curlOutput)) then
    DeleteFile(InstallerPath + curlOutput);  
  curlOutputTmp := curlOutput + '.tmp';
  if (fileExists(InstallerPath + curlOutputTmp)) then
    DeleteFile(InstallerPath + curlOutputTmp);  
    
  sCommand := '-L --output "' + InstallerPath + curlOutputTmp + '" --url "' + address + '" ' + '-H "Accept: text/html, */*" -H "Accept-Language: it" -H "DNT: 1" -H "Priority: u=0, i" -H "Sec-Ch-Ua: \"Not)A;Brand\";v=\"8\", \"Chromium\";v=\"138\", \"Google Chrome\";v=\"138\"" -H "Sec-Ch-Ua-Mobile: ?0" -H "Sec-Ch-Ua-Platform: \"Windows\"" -H "Sec-Fetch-Dest: Document" -H "Sec-Fetch-Mode: Navigate" -H "Sec-Fetch-Site: None" -H "Sec-Fetch-User: ?1" -H "Upgrade-Insecure-Requests: 1" -H "User-Agent: ' + curlUserAgent + '"';

  Sleep(delayBetweenRequest);
  Launch(curlPath, sCommand); 
    
  cnt := 0;
  while ((not MoveFile(InstallerPath + curlOutputTmp, InstallerPath + curlOutput)) AND (cnt < 200)) Do
  begin
    cnt := cnt + 1;
    Sleep(50);
  end;  
  if (fileExists(InstallerPath + curlOutput)) then
  begin
    // wait until file is ready (you don't really need this)
    Sleep(5);    
    fileContent := TStringList.Create;
    fileContent.LoadFromFile(InstallerPath + curlOutput);
    Result := fileContent.Text;
    fileContent.Free;
    DeleteFile(InstallerPath + curlOutput);
  end;
end;

end.

Me voy de vacaciones y a la vuelta veo que hay movimiento, ¡¡fantástico !!
Muchas gracias Garada, da tranquilidad que haya alguien más implicado del nivel de Antp, MrObama2022, etc...

He probado tú código y va bien, aunque en mi caso como comenta MrObama2022 se ve durante unos instantes la ventana CMD. Ningún problema por mi parte que se vea, lo que decidáis me parece bien.

Muchas gracias a todos!!!

[Radagast]

Si a alguien por casualidad le deja de funcionar el script en estas 2 situaciones.
Con la versión portable de AMC en un pendrive y al conectarlo cambia la letra de unidad asignada la última vez, o cambia la ubicación de la carpeta scripts en un AMC instalado.
Para que vuelva a funcionar es necesario borrar el archivo ExternalCurlHandler.vbs ubicado en la carpeta scripts para que se vuelva a generar.

Esto es para la versión de MrObama2022, a la versión de Garada no le afecta al llamar directamente a curl.

[Radagast]

NOTE 2:

To avoid problems such as PACOIRIPO's, I would change the way the VBS file is executed.

Try this (not tested):

Code: Select all

Launch('wscript.exe', InstallerPath + vbsScript);

He probado a cambiar la linea

Code: Select all

Launch(InstallerPath + vbsScript, '');

por la que comentas.
En la versión instalada de AMC funciona bien.
En la versión portable solo me funciona bien si el path donde está AMCportable no contiene espacios.
Si contiene espacios sale una ventana de error Windows Script Host que dice No existe ninguna extension de archivo en "I:\Ant", el path donde está AMCportable es I:\Ant Movie Catalog (portable)\. Es como si los espacios evitaran que se cogiera el path completo, pero solo en la versión portable.
Si utilizo la linea original de MrObama2022 o elimino los espacios del path de AMCportable funciona bien

[Garada]

Sí, efectivamente VBS maneja diferente las comillas dobles.

Yo uso el AMC instalado y no he pensado en la versión portable

Te dejo por aquí la última versión que he preparado que incluye todas las ideas de MrObama2022 ha ido comentando e intentando simplificar el código.

Si la pruebas, debes borrar el archivo VBS para que lo cree de nuevo ya que es un código diferente. Esta versión no tiene a fuego la llamada a CURL sino que recibe la llamada por parámetros de la función principal. De esa forma no debería afectarle el cambio de la carpeta de los scripts.

También dará mensajes de error que ayudarán a afinar el script, cada equipo es un mundo.

Code: Select all

unit ExternalCurlHandler;

uses StringUtils7552;

const
  tmpDir = ''; // (optional) if you want get your scripts directory clean, set here your tmp working dir, example C:\Users\YOURWINDOWSUSER\AppData\Local\Temp\ . Remember to add \ at the end of the directory
  delayBetweenRequest = 2001;
  curlPath = 'curl.exe'; // if you use Windows 7 or 8 download curl.exe for Windows (it's free) and set here the right path
  vbsScript = 'ExternalCurlHandler.vbs';
  curlOutput = 'curlOutput.html';
  curlUserAgent = 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/138.0.0.0 Safari/537.36';
  TimeOut = 10000; // Time to wait for response in ms

var
  InstallerPath: string;
  
function GetPage5Advanced(address: string; referer: string; cookies: string; content: string; headers: string): string;
var
  cnt: integer;
  fileContent: TStringList;
  curlOutputResult: string;
  sCommand: string;
begin
  Result := '';
  
  // Create VBS file if not exists
  if setupScript then
  begin
    // Delete temporal files
    if fileExists(InstallerPath + curlOutput) then
      DeleteFile(InstallerPath + curlOutput);  
    curlOutputResult := curlOutput + '.result';
    if fileExists(InstallerPath + curlOutputResult) then
      DeleteFile(InstallerPath + curlOutputResult);  
    
    // CURL parameters:
    // Create info file with return codes and possible errors, it's created after page is downloaded
    sCommand := '-w "%output{' + InstallerPath + curlOutputResult + '}%{url}\nExitCode: %{exitcode}\nErrorMsg: %{errormsg}\nResponseCode: %{http_code}"';
    // Download page and save to file
    sCommand := sCommand + ' -L --output "' + InstallerPath + curlOutput + '" --url "' + address + '" ' + '-H "Accept: text/html, */*" -H "Accept-Language: it" -H "DNT: 1" -H "Priority: u=0, i" -H "Sec-Ch-Ua: \"Not)A;Brand\";v=\"8\", \"Chromium\";v=\"138\", \"Google Chrome\";v=\"138\"" -H "Sec-Ch-Ua-Mobile: ?0" -H "Sec-Ch-Ua-Platform: \"Windows\"" -H "Sec-Fetch-Dest: Document" -H "Sec-Fetch-Mode: Navigate" -H "Sec-Fetch-Site: None" -H "Sec-Fetch-User: ?1" -H "Upgrade-Insecure-Requests: 1" -H "User-Agent: ' + curlUserAgent + '"';
    //test slow connection, don't uncomment
    //sCommand := '--limit-rate 1 ' + sCommand;

    Sleep(delayBetweenRequest);
    // Launch CURL. Change " for #1 as VBS uses " for parameters. The VBS script undone the changes.
    Launch('wscript.exe', '"' +InstallerPath + vbsScript + '" "' + curlPath + '" ' + StringReplace(sCommand, '"', #1)); 
    
    // Wait for end info file or timeout
    cnt := 0;
    while (not FileExists(InstallerPath + curlOutputResult)) and (cnt < TimeOut div 50) do
    begin
      cnt := cnt + 1; 
      Sleep(50);
    end;  
    
    // if info file exists
    if (fileExists(InstallerPath + curlOutputResult)) then
    begin
      fileContent := TStringList.Create;
      try
        // Read and delete info file 
        fileContent.LoadFromFile(InstallerPath + curlOutputResult); 
        DeleteFile(InstallerPath + curlOutputResult);
        
        // if return error
        if TextBetween(fileContent.Text, 'ErrorMsg: ', #13) <> '' then
          ShowError('Error downloading page.' + #13 + fileContent.Text)
        else if (fileExists(InstallerPath + curlOutput)) then // if downloaded page exits
        begin
          // Read and delete downloaded page
          fileContent.LoadFromFile(InstallerPath + curlOutput);
          DeleteFile(InstallerPath + curlOutput);
          // Return page
          Result := fileContent.Text;
          
          //if Pos('</html>', Result) < 1 then
          //  ShowError('TRIM!!!');
        end
        else // no error, no download, no timeout...
          ShowError('The page did not download');
      finally
        fileContent.Free;
      end;
    end
    else // if not: timeout
      ShowError('Internal Timeout!!');
  end
  else
  begin
    Sleep(delayBetweenRequest);
    Result := GetPage5(address, referer, cookies, content, headers);
  end;  
end;


function setupScript: boolean;
var 
  ScriptContent: TStringList;
begin
  Result := False;
  
  // initialize working path
  if (tmpDir <> '') then
    InstallerPath := tmpDir
  else  
    InstallerPath := dirScripts;
  
  // Create a generic VBS script that
  // open a command with parameters in hidden window
  if (not FileExists(InstallerPath + vbsScript)) then
  begin
    ScriptContent := TStringList.Create;
    ScriptContent.Add('Dim Args()');
    ScriptContent.Add('ReDim Args(WScript.Arguments.Count - 1)');
    ScriptContent.Add('Args(0) = """" & WScript.Arguments(0) & """"');
    ScriptContent.Add('For i = 1 To WScript.Arguments.Count - 1');
    ScriptContent.Add('   Args(i) = Replace(WScript.Arguments(i), chr(1), chr(34))');
    ScriptContent.Add('Next');
    ScriptContent.Add('CreateObject("Wscript.Shell").Run Join(Args), 0, False');
    ScriptContent.SaveToFile(InstallerPath + vbsScript); 
    ScriptContent.Free;   
  end;

  Result := true;  
end;  

begin
end.

[Radagast]

Sí, efectivamente VBS maneja diferente las comillas dobles.

Yo uso el AMC instalado y no he pensado en la versión portable

Te dejo por aquí la última versión que he preparado que incluye todas las ideas de MrObama2022 ha ido comentando e intentando simplificar el código.

Si la pruebas, debes borrar el archivo VBS para que lo cree de nuevo ya que es un código diferente. Esta versión no tiene a fuego la llamada a CURL sino que recibe la llamada por parámetros de la función principal. De esa forma no debería afectarle el cambio de la carpeta de los scripts.

También dará mensajes de error que ayudarán a afinar el script, cada equipo es un mundo.

Probado con la versión portable desde un pendrive en el ordenador del trabajo y sin derechos de administrador. El path con espacios, ya no da problemas.
He actualizado un grupo de 20 películas lo más rápido que he podido con delayBetweenRequest = 501 sin problemas, todo como una seda.
Cuando pueda lo haré en casa con la versión instalada.

Me he dado cuenta que ya no crea (o bien lo borra) el archivo ExternalCurlHandler.ps1.

Gran trabajo Garada