feat: change to diff workflow

README.md

@@ -1,5 +1,5 @@
 <!-- README.md -->
-# ChatGPT NeoVim Plugin (Extensively Updated with Step-by-Step Prompting)
+# ChatGPT NeoVim Plugin (Extensively Updated with Step-by-Step Prompting and Diff-Based Changes)
 
 This plugin integrates a ChatGPT O1 model workflow into Neovim. It allows you to:
 
@@ -10,21 +10,26 @@ This plugin integrates a ChatGPT O1 model workflow into Neovim. It allows you to
    - Any **initial files** you define (e.g., `README.md`, etc.)
 
 2. Copy these prompts to your clipboard to paste into ChatGPT O1.
-3. Receive YAML changes from ChatGPT, then run `:ChatGPTPaste` to apply them or supply additional files.
+3. Receive **YAML changes from ChatGPT that include diffs**, then run `:ChatGPTPaste` to apply them or supply additional files.  
+   - If you’re updating an existing file, provide a `diff` field in the YAML.  
+   - If you’re creating a new file, use the `content` field.  
+   - If you’re deleting a file, use `delete: true`.  
 
 ## New Key Features
 
 - **Step-by-Step Prompting** (`enable_step_by_step: true`):  
-  If the request grows too large (exceeds `token_limit`), the plugin automatically generates a special prompt asking the model to split the task into smaller steps, working through them one by one. This approach helps you stay within the model’s maximum token limit without having to manually break things apart.
+  If the request grows too large (exceeds `token_limit`), the plugin automatically generates a special prompt asking the model to split the task into smaller steps, working through them one by one.
 
-- **Partial Acceptance**: If `partial_acceptance: true`, you can open a buffer that lists the final changes. Remove or comment out lines you don’t want, then only those changes are applied.
+- **Partial Acceptance**: If `partial_acceptance: true`, you can open a buffer that lists final diffs or file creations. You can remove or comment out lines you don’t want, then only those changes are applied.
 
-- **Preview Changes**: If `preview_changes: true`, you get a buffer showing proposed changes before you apply them.
+- **Preview Changes**: If `preview_changes: true`, you get a buffer showing proposed diffs or new file content before you apply them.
 
 - **Interactive File Selection**: If `interactive_file_selection: true`, you choose which directories from `.chatgpt_config.yaml` get included in the prompt, reducing token usage.
 
 - **Improved Debug**: If `improved_debug: true`, debug logs go into a dedicated `ChatGPT_Debug_Log` buffer for easier reading.
 
+- **Diff-Based Changes**: Rather than supplying entire file content for edits, you can include a `diff` in the YAML response. This allows you to see exactly what changed line by line and accept or reject it.
+
 ## Example `.chatgpt_config.yaml`
 
 ```yaml
@@ -49,7 +54,7 @@ token_limit: 3000
 
 1. **`:ChatGPT`**  
    - If `interactive_file_selection` is on, you’ll pick directories to include in a buffer named `ChatGPT_File_Selection`.
-   - Save & close with `:wq`, `:x`, or `:bd` (you don’t have to use `:q`).
+   - Save & close with `:wq`, `:x`, or `:bd`.
    - If `enable_step_by_step` is on and the prompt might exceed `token_limit`, the plugin will generate instructions prompting the model to address each step separately.
 
 2. **Paste Prompt to ChatGPT**  
@@ -60,7 +65,7 @@ token_limit: 3000
    - If final changes are provided:
      - Optionally preview them (`preview_changes`).  
      - Optionally partially accept them (`partial_acceptance`).  
-     - Then the plugin writes/deletes files as specified.
+     - The plugin then applies file creation or deletion, or applies diffs to existing files.
 
 ## Troubleshooting & Tips
 - Adjust `token_limit` in `.chatgpt_config.yaml` as needed.
@@ -80,6 +85,6 @@ commands:
     pattern: "searchString"
     target: "path/to/file/or/directory"
 ```
-The **list** command now uses the Linux `ls` command to list directory contents. The **grep** command searches for a given pattern in a file or all files in a directory.
+The **list** command uses the system's `ls` command to list directory contents. The **grep** command searches for a given pattern in a file or all files in a directory.
 
-Enjoy your improved, more flexible ChatGPT Neovim plugin with step-by-step support!
+Enjoy the improved, more flexible ChatGPT Neovim plugin with step-by-step and diff-based support!

lua/chatgpt_nvim/config.lua

@@ -6,9 +6,7 @@ local ok_yaml, lyaml = pcall(require, "lyaml")
 local prompt_blocks = {
   ["go-development"] = [[
     You are a coding assistant specialized in Go development.
-    You will receive a project’s context and user instructions related to Go code,
+    You will receive a project’s context and user instructions related to Go code.
-    and you must return the requested modifications or guidance.
-    When returning modifications, follow the specified YAML structure.
     Keep your suggestions aligned with Go best practices and idiomatic Go.
   ]],
   ["typo3-development"] = [[
@@ -19,51 +17,36 @@ local prompt_blocks = {
   ]],
   ["rust-development"] = [[
     You are a coding assistant specialized in Rust development.
-    You will receive a project’s context and user instructions related to Rust code,
+    You will receive a project’s context and user instructions related to Rust code.
-    and you must return the requested modifications or guidance.
-    When returning modifications, follow the specified YAML structure.
 	  Keep your suggestions aligned with Rust best practices and idiomatic Rust.
   ]],
   ["basic-prompt"] = [[
     You are a coding assistant who receives a project's context and user instructions.
     The user will provide a prompt, and you will guide them through a workflow:
-    1. First, you should analyse which files you need to solve the request.
+    1. Analyse which files you need. Ask for file contents in YAML if needed.
-      You can see which files are present in the provided project structure.
+    2. Request additional context outside YAML if necessary.
-      Additionally if presented you could also ask for files of a library which is provided in for example composer.json.
+    3. When ready, provide final changes in YAML with:
-    2. If file contents is needed provide a yaml which asks for the file contents.
+       - 'project_name'
-      For example:
+       - 'files', each having:
-      project_name: example_project
+         * 'path'
-      files:
+         * 'diff' (for patching an existing file) OR 'content' (for a new file) OR 'delete'
-      - path: "relative/path/to/file"
+    Important: do not provide entire file content for updates; instead provide a unified diff in 'diff'.
-    3. If more information or context is needed, ask the user (outside of the YAML) to provide that.
+    Only modify or delete files whose contents you have explicitly requested and seen beforehand.
-    4. When all necessary information is gathered, provide the final YAML with the
-      project's name and a list of files to be created or modified.
-      Also explain the changes you made below the yaml.
-
-    The final YAML must have a top-level key named 'project_name' that matches the project's configured name,
-    and a top-level key named 'files', which is a list of file changes. Each element in 'files' must be a mapping with:
-    - 'path' for the file path relative to the project’s root directory.
-    - either 'content' with a multiline string for new content, or 'delete: true' if the file should be deleted.
-    Important: dont use comments in the code to explain which steps you have taken.
-    Comments should just explain the code and not your thought process.
-    You can explain your thought process outside of the YAML.
-
-    If more context is needed at any point before providing the final YAML, request it outside of the YAML.
-    Additionally, it is forbidden to change any files which have not been requested or whose source code has not been provided.
   ]],
   ["secure-coding"] = [[
     You are a coding assistant specialized in secure software development.
-    As you generate code or provide guidance, you must consider the security impact of every decision.
+    Always consider security impacts. Use diffs for updates, new content for new files,
-    You will write and review code with a focus on minimizing vulnerabilities and following best security practices,
+    and 'delete: true' for removals.
-    such as validating all user inputs, avoiding unsafe libraries or functions, and following secure coding standards.
   ]],
   ["workflow-prompt"] = [[
     You are a coding assistant focusing on making the Neovim ChatGPT workflow straightforward and user-friendly.
-    Provide a concise set of steps or guidance, reminding the user:
+    Remind the user to:
-    - How to list needed files for further context
+    - List needed files for further context
-    - How to request additional information outside of the YAML
+    - Request additional information outside YAML if needed
-    - How to finalize changes with a YAML response containing project_name and files
+    - Provide final changes in YAML with 'project_name' and 'files', using:
-    Always ensure that prompts and explanations remain clear and minimal, reducing user errors.
+      * 'diff' for existing file modifications
+      * 'content' for new files
+      * 'delete: true' for file deletions
   ]]
 }
 
@@ -103,7 +86,6 @@ function M.load()
     initial_prompt = "",
     directories = { "." },
     default_prompt_blocks = {},
-    -- Changed default from 128000 to 16384 as requested
     token_limit = 16384,
     project_name = "",
     debug = false,
@@ -113,7 +95,6 @@ function M.load()
     partial_acceptance = false,
     improved_debug = false,
     enable_chunking = false,
-    -- New default for step-by-step
     enable_step_by_step = true,
     enable_debug_commands = false
   }
@@ -161,7 +142,6 @@ function M.load()
         if type(result.enable_chunking) == "boolean" then
           config.enable_chunking = result.enable_chunking
         end
-        -- Added logic to load enable_step_by_step from user config
         if type(result.enable_step_by_step) == "boolean" then
           config.enable_step_by_step = result.enable_step_by_step
         end
@@ -174,7 +154,6 @@ function M.load()
     config.initial_prompt = "You are a coding assistant who receives a project's context and user instructions..."
   end
 
-  -- Merge the default prompt blocks with the config's initial prompt
   if type(config.default_prompt_blocks) == "table" and #config.default_prompt_blocks > 0 then
     local merged_prompt = {}
     for _, block_name in ipairs(config.default_prompt_blocks) do

lua/chatgpt_nvim/handler.lua

@@ -69,6 +69,76 @@ function M.delete_file(filepath)
   end
 end
 
+-- Applies a unified diff to the specified file.
+-- This spawns an external 'patch' command if available.
+function M.apply_diff(filepath, diff_content)
+  local conf = config.load()
+  local tmp_original = vim.fn.tempname()
+  local tmp_patch = vim.fn.tempname()
+
+  -- Read original file (or empty if it doesn't exist)
+  local fd_in = uv.fs_open(filepath, "r", 438)
+  local original_data = ""
+  if fd_in then
+    local stat = uv.fs_fstat(fd_in)
+    original_data = uv.fs_read(fd_in, stat.size, 0)
+    uv.fs_close(fd_in)
+  end
+
+  -- Write original content to temp file
+  local fd_orig = uv.fs_open(tmp_original, "w", 438)
+  if fd_orig then
+    uv.fs_write(fd_orig, original_data, -1)
+    uv.fs_close(fd_orig)
+  end
+
+  -- Write diff to temp file
+  local fd_patch = uv.fs_open(tmp_patch, "w", 438)
+  if fd_patch then
+    uv.fs_write(fd_patch, diff_content, -1)
+    uv.fs_close(fd_patch)
+  else
+    return false, "Could not open temporary file to write patch."
+  end
+
+  -- Attempt to run 'patch'
+  local patch_cmd = "patch -u " .. vim.fn.shellescape(tmp_original) .. " < " .. vim.fn.shellescape(tmp_patch)
+  local handle = io.popen(patch_cmd)
+  if not handle then
+    return false, "Failed to run patch command."
+  end
+  local result = handle:read("*a")
+  local success_close, errmsg = handle:close()
+
+  if conf.debug then
+    vim.api.nvim_out_write("[chatgpt_nvim:handler] Patch command output:\n" .. (result or "") .. "\n")
+  end
+
+  if not success_close then
+    if conf.debug then
+      vim.api.nvim_out_write("[chatgpt_nvim:handler] Patch command failed: " .. (errmsg or "unknown") .. "\n")
+    end
+    return false, errmsg
+  end
+
+  -- If successful, read the patched file and write it back
+  local fd_out = uv.fs_open(tmp_original, "r", 438)
+  if fd_out then
+    local stat_out = uv.fs_fstat(fd_out)
+    local new_data = uv.fs_read(fd_out, stat_out.size, 0)
+    uv.fs_close(fd_out)
+
+    M.write_file(filepath, new_data)
+    if conf.debug then
+      vim.api.nvim_out_write("[chatgpt_nvim:handler] Successfully applied patch to: " .. filepath .. "\n")
+    end
+  else
+    return false, "Could not read patched file."
+  end
+
+  return true
+end
+
 function M.finish()
   print("Finished processing files.")
 end

lua/chatgpt_nvim/init.lua

@@ -111,18 +111,31 @@ local function preview_changes(changes)
     ""
   })
   for _, fileinfo in ipairs(changes) do
-    local indicator = (fileinfo.delete == true) and "Delete file" or "Write file"
+    if fileinfo.delete then
       vim.api.nvim_buf_set_lines(bufnr, -1, -1, false, {
-      string.format("=== %s: %s ===", indicator, fileinfo.path or "<no path>")
+        string.format("=== Delete file: %s ===", fileinfo.path or "<no path>"),
+        ""
+      })
+    elseif fileinfo.diff then
+      vim.api.nvim_buf_set_lines(bufnr, -1, -1, false, {
+        string.format("=== Diff for file: %s ===", fileinfo.path or "<no path>")
+      })
+      local lines = vim.split(fileinfo.diff, "\n")
+      for _, line in ipairs(lines) do
+        vim.api.nvim_buf_set_lines(bufnr, -1, -1, false, { line })
+      end
+      vim.api.nvim_buf_set_lines(bufnr, -1, -1, false, { "" })
+    elseif fileinfo.content then
+      vim.api.nvim_buf_set_lines(bufnr, -1, -1, false, {
+        string.format("=== New file: %s ===", fileinfo.path or "<no path>")
       })
-    if fileinfo.content then
       local lines = vim.split(fileinfo.content, "\n")
       for _, line in ipairs(lines) do
         vim.api.nvim_buf_set_lines(bufnr, -1, -1, false, { line })
       end
-    end
       vim.api.nvim_buf_set_lines(bufnr, -1, -1, false, { "" })
     end
+  end
 
   vim.cmd("vsplit")
   vim.cmd("buffer " .. bufnr)
@@ -139,9 +152,16 @@ local function partial_accept(changes)
     ""
   }
   for _, fileinfo in ipairs(changes) do
-    local action = (fileinfo.delete == true) and "[DELETE]" or "[WRITE]"
+    if fileinfo.delete then
-    table.insert(lines, string.format("%s %s", action, fileinfo.path or "<no path>"))
+      table.insert(lines, string.format("[DELETE] %s", fileinfo.path or "<no path>"))
-    if fileinfo.content then
+    elseif fileinfo.diff then
+      table.insert(lines, string.format("[DIFF] %s", fileinfo.path or "<no path>"))
+      local diff_lines = vim.split(fileinfo.diff, "\n")
+      for _, dl in ipairs(diff_lines) do
+        table.insert(lines, "  " .. dl)
+      end
+    elseif fileinfo.content then
+      table.insert(lines, string.format("[WRITE] %s", fileinfo.path or "<no path>"))
       local content_lines = vim.split(fileinfo.content, "\n")
       for _, cl in ipairs(content_lines) do
         table.insert(lines, "  " .. cl)
@@ -156,46 +176,53 @@ local function partial_accept(changes)
   local function on_write()
     local edited_lines = vim.api.nvim_buf_get_lines(bufnr, 0, -1, false)
     local keep_current = false
-    local current_fileinfo = { path = nil, content = nil, delete = false }
+    local current_fileinfo = { path = nil, delete = false, diff = nil, content = nil }
-    local content_accum = {}
+    local accum = {}
 
     for _, line in ipairs(edited_lines) do
       if line:match("^#") or line == "" then
         goto continue
       end
+
       local del_match = line:match("^%[DELETE%] (.+)")
+      local diff_match = line:match("^%[DIFF%] (.+)")
       local write_match = line:match("^%[WRITE%] (.+)")
+
+      if del_match or diff_match or write_match then
+        -- store previous if any
+        if keep_current and (current_fileinfo.path ~= nil) then
+          if current_fileinfo.diff then
+            current_fileinfo.diff = table.concat(accum, "\n")
+          elseif current_fileinfo.content then
+            current_fileinfo.content = table.concat(accum, "\n")
+          end
+          table.insert(final_changes, current_fileinfo)
+        end
+
+        accum = {}
+        keep_current = true
+
         if del_match then
-        if keep_current and (current_fileinfo.path ~= nil) then
+          current_fileinfo = { path = del_match, delete = true, diff = nil, content = nil }
-          if #content_accum > 0 then
+        elseif diff_match then
-            current_fileinfo.content = table.concat(content_accum, "\n")
+          current_fileinfo = { path = diff_match, delete = false, diff = "", content = nil }
-          end
-          table.insert(final_changes, current_fileinfo)
-        end
-        keep_current = true
-        current_fileinfo = { path = del_match, delete = true, content = nil }
-        content_accum = {}
         elseif write_match then
-        if keep_current and (current_fileinfo.path ~= nil) then
+          current_fileinfo = { path = write_match, delete = false, diff = nil, content = "" }
-          if #content_accum > 0 then
-            current_fileinfo.content = table.concat(content_accum, "\n")
         end
-          table.insert(final_changes, current_fileinfo)
+
-        end
-        keep_current = true
-        current_fileinfo = { path = write_match, delete = false, content = nil }
-        content_accum = {}
       else
         if keep_current then
-          table.insert(content_accum, line:gsub("^%s*", ""))
+          table.insert(accum, line:gsub("^%s*", ""))
         end
       end
       ::continue::
     end
 
     if keep_current and (current_fileinfo.path ~= nil) then
-      if #content_accum > 0 then
+      if current_fileinfo.diff ~= nil then
-        current_fileinfo.content = table.concat(content_accum, "\n")
+        current_fileinfo.diff = table.concat(accum, "\n")
+      elseif current_fileinfo.content ~= nil then
+        current_fileinfo.content = table.concat(accum, "\n")
       end
       table.insert(final_changes, current_fileinfo)
     end
@@ -399,7 +426,7 @@ function M.run_chatgpt_command()
 
           ```yaml
           commands:
-            - command: "ls"
+            - command: "list"
               dir: "some/directory"
 
             - command: "grep"
@@ -407,7 +434,6 @@ function M.run_chatgpt_command()
               target: "path/to/file/or/directory"
           ```
 
-          The "ls" command uses the system's 'ls' command to list directory contents.
           When these commands are present and enable_debug_commands is true, I'll execute them and return the results in the clipboard.
         ]])
       end
@@ -467,7 +493,7 @@ function M.run_chatgpt_paste_command()
 
     local is_final = false
     for _, fileinfo in ipairs(data.files) do
-      if fileinfo.content or fileinfo.delete == true then
+      if fileinfo.content or fileinfo.delete == true or fileinfo.diff then
         is_final = true
         break
       end
@@ -518,12 +544,20 @@ function M.run_chatgpt_paste_command()
           ui.debug_log("Deleting file: " .. fileinfo.path)
           handler.delete_file(fileinfo.path)
           print("Deleted: " .. fileinfo.path)
+        elseif fileinfo.diff then
+          ui.debug_log("Applying diff to file: " .. fileinfo.path)
+          local success, err = handler.apply_diff(fileinfo.path, fileinfo.diff)
+          if not success then
+            vim.api.nvim_err_writeln("Error applying diff: " .. (err or "unknown"))
+          else
+            print("Patched: " .. fileinfo.path)
+          end
         elseif fileinfo.content then
-          ui.debug_log("Writing file: " .. fileinfo.path)
+          ui.debug_log("Writing new file: " .. fileinfo.path)
           handler.write_file(fileinfo.path, fileinfo.content)
           print("Wrote: " .. fileinfo.path)
         else
-          vim.api.nvim_err_writeln("Invalid file entry. Must have 'content' or 'delete'.")
+          vim.api.nvim_err_writeln("Invalid file entry. Must have 'diff', 'content', or 'delete'.")
         end
         ::continue::
       end
@@ -553,7 +587,7 @@ function M.run_chatgpt_paste_command()
         "\n\nProject name: " .. (conf.project_name or ""),
         "\n\nBelow are the requested files from the project, each preceded by its filename in backticks and enclosed in triple backticks.\n",
         table.concat(file_sections, "\n"),
-        "\n\nIf you need more files, please respond again in YAML listing additional files. If you have all information you need, provide the final YAML with `project_name` and `files` (with `content` or `delete`) to apply changes.\n"
+        "\n\nIf you need more files, please respond again in YAML listing additional files. If you have all information you need, provide the final YAML with `project_name` and `files` (use `diff`, `content`, or `delete`) to apply changes.\n"
       }
 
       local prompt = table.concat(sections, "\n")
@@ -647,7 +681,6 @@ function M.run_chatgpt_current_buffer_command()
           target: "path/to/file/or/directory"
       ```
 
-      The "list" command uses the system's 'ls' command to list directory contents.
       When these commands are present and enable_debug_commands is true, I'll execute them and return the results in the clipboard.
     ]])
   end