runtime(zip): Customizable zip/unzip commands - #20832

this also requires an update to runtime/doc/pi_zip.txt. Also please be very careful, to not introduce command injection or arbitrary file writes. We had some security issues around this.

Ok. This pr makes no difference to filename logic. I added more custom commands, and every default value of them should be checked by dist#vim#IsSafeExecutable prevent injection by environment, as before.

The default commands use zip and unzip, and I tried using 7z (note that the -ba option of 7z is not stable API):

let g:zip_browse= ["printf", "'';",
  \ "7zlbaf() { 7z l -ba \"$@\" | awk '{print ($3 ~ /D[^[:space:]]{4}/ ? $6 \"/\" : $6)}'; }",
  \ " && ", "7zlbaf"]    " or save command as a file with `#!/bin/sh`
let g:zip_read= ["7z", "x", "-so"]
let g:zip_extract= ["7z", "x"]
let g:zip_delete= ["7z", "d"]
let g:zip_update= ["7z", "u"]

I found that 7z behaviors different on file names with special characters, such as a[a].txt or a*.txt. For example:

$ ls zipglob
'a[a].txt'  'a*.txt'  'a?.txt'  'a\.txt'  'a\\.txt'
$ wc -l 'zipglob/a*.txt'
1 zipglob/a*.txt
$ unzip -p exp.zip 'zipglob/a\*.txt' | wc -l
1
$ 7z x -so exp.zip 'zipglob/a\*.txt' | wc -l
2

This means the behavior of visiting(read/write) files that contains special character in a zip file without zip/unzip is unpredictable.

This means the behavior of visiting(read/write) files that contains special character in a zip file without zip/unzip is unpredictable.

Not only that, the zip/unzip command line parser is already insane. There is no sane way to handle strange characters robustly.

That would mean we would need a custom escape function like what is done for powershell using PSEscape().

I found there's no support for Windows PowerShell 5, only PowerShell Core. So the CI test fails. I'll fix it.

Not only that, the zip/unzip command line parser is already insane. There is no sane way to handle strange characters robustly.

That would mean we would need a custom escape function like what is done for powershell using PSEscape().

When the commands to open/edit zip files are customizable, the resposibility to guarantee the correctness are transmit to the users. Based on this opinion, us developers should ensure the default behaviors are correct.

As for my above example:

let g:zip_browse= ["printf", "'';",
  \ "7zlbaf() { 7z l -ba \"$@\" | awk '{print ($3 ~ /D[^[:space:]]{4}/ ? $6 \"/\" : $6)}'; }",
  \ " && ", "7zlbaf"]

This hack is not good since one more meanless command is executed. There's another example (on Windows):

let g:zip_browse  = ["7zlbaf"]
let g:zip_read    = ["7z", "x", "-so"]
let g:zip_extract = ["7z", "x"]
let g:zip_delete  = ["7z", "d"]
let g:zip_update  = ["7z", "u"]

while the 7zlbaf.bat is a custom script file which is added to %PATH%(or $Env:Path), with awk installed:

@7z l -ba %* | awk.exe "{ gsub(/\\/, \""/\"", $6); print ($3 ~ /D[^[:space:]]{4}/ ? $6 \""/\"" : $6)}"

They are still not elegant. Maybe we should provide an API to let users to format the outputs of commands, which may solve special character problem.

so you think this is worthwhile? or shall we rather drop it?

so you think this is worthwhile? or shall we rather drop it?

I thnk at least (runtime(zip): Compatability for both powershell 5 and pwsh 7) is meaningful.

@7z l -ba %* | awk.exe "{ gsub(/\\/, \""/\"", $6); print ($3 ~ /D[^[:space:]]{4}/ ? $6 \""/\"" : $6)}"

They are still not elegant. Maybe we should provide an API to let users to format the outputs of commands, which may solve special character problem.

I tested providng a new API on my device (which is not committed):

if !exists("g:zip_browse_format")
  let g:zip_browse_tailcmd= [""]
endif

" ...

  let gnu_cmd = 'keepj sil r! ' . join(g:zip_browse) . ' ' . s:Escape(a:zipfile, 1) . ' ' . join(g:zip_browse_tailcmd)

And the configuration becomes to(on Windows):

let g:zip_browse  = ["7z l -ba"]
let g:zip_browse_format  = ["|", "awk", "'" . '{ gsub(/\\/, \\\"/\\\", $6); print ($3 ~ /D[^[:space:]]{4}/ ? $6 \\\"/\\\" : $6) }' . "'"]

It's kind of painful to debug to right codes because of escape characters. Or provide a customizable function g:zip_browse_format to format outputs, but this means manipulting strings with vimscript; and cannot make use of stream mechanism of pipe, which is more efficient.

So do not provide this API is better. Users can do some hack on the new global variables (g:zip_browse, ...). Examples see my comments above.

Okay. I merged in untime(zip): Compatability for both powershell 5 and pwsh 7 and fixed a few bugs in that commit. I also included the change for the SafeExecutable().

The thing I do not like about the new variables is, that you are changing the default types to a list. That will bite each user who has been customizing those variables, so I left that out.

Have a look at it and see if this works for you. Thanks!

The thing I do not like about the new variables is, that you are changing the default types to a list. That will bite each user who has been customizing those variables, so I left that out.

I'm glad to know your attitude to destructive changes. But my original purpose is to introduce customizable zip/unzip commands. I found a hard encoding issue, then found this API is not reasonable.

If anyone configure to let g:zip_cmd = '7z' the gnu_cmd would be keepj sil r! 7z -Z1 -- FPATH, which always fails. This means the original API exposure is unavailable, or useless. So I introduced the destructive changes.

If destructive changes are not expected, we can cosider another way to implement the functionality. For example, expose more variables.

Well, the other alternative would be to use vim variarbles, that are set by the existing variables if those exists, something like this:

if !exists('g:zip_zipcmd_list')
  if exists('g:zip_zipcmd')
    let g:zip_zipcmd_list = [g:zip_zipcmd, '-d']
  else
    let g:zip_zipcmd_list = ['zip', '-d']
  endif
endif