OpenMage coding with AI tools?

[colinmollenhour]

Just curious if anyone has had much success with AI coding tools? I've played around some with:

• PhpStorm + Github Copilot
  • This is what I'm using currently, but I don't use it much. Occasionally the autocompletion suggestion is magic, but often it hallucinates, so it's a bit distracting.
• PhpStorm + ClaudeMind
  • The AI was pretty good, but the plugin is too buggy and I hit too many rate limits on my Anthropic API key.
• VSCode + Cline (with Claude 3.5 Sonnet)
  • Seemed promising, but I would run into API rate limits on my Anthropic API key in the middle of it working which is frustrating.
• Continue.dev and other VS Code forks
  • I haven't tried these recently, but they seemed to not be any better than VS Code plus an extension.

The one promising thing I haven't tried yet is Aider + Deepseek. With Deepseek's large context limit and Aider's ability to cache context and refine what files are included using .aiderignore I think this could work really well. However, the learning curve is a bit steeper so I've not been able to invest enough time to set it up and try it out properly.

Overall, it seems that for small and completely new projects like a Node.js microservice there is a ton of potential, but the OpenMage codebase is too large to have AI write large chunks of good code, and it's mostly the autocomplete that can save you a few seconds here and there.. Has anyone else had better luck getting AI to boost your productivity when coding with OpenMage?

[fballiano]

IMHO there are multiple aspect at play here:

• magento's lack of documentation -> training of the models is not that great
• stackoverflow super old content, many times suggesting wrong or outdated practices -> poor LLMs suggestions
• large codebase -> IDE AI plugins can't handle it

If you sum all of the above... you get a lot of bad AI suggestions, which IMHO are much worse than not having the suggestion at all.

So, for the time being, I disable all AI feature in the IDE, and manually ask stuff to Claude/R1 that I review and copy/paste myself. Not a great experience but worth it sometimes.

[colinmollenhour]

Here's an example of using Aider to create some simple models. My .aiderignore file looks like this:

/*
!app/code/core/

And to keep the context small, I used this to run aider:

cd app/code/core/Mage/Customer
aider --subtree-only

Then here is my session (using Claude 3.5 Sonnet):

2025-02-07_18-34-48.mp4

So it depends on what you're doing. If tweaking the .aiderignore a bit and running it from different subtrees gets you good results and is worth your time, it seems like it's getting pretty promising as far as using context effectively without being overwhelmed. This test cost $0.09. You could probably burn through a couple bucks a day easily at that rate, but Deepseek is a lot cheaper.

So it may be that an effective strategy is to use different .aiderignore files that are specific to the context you need at the time.

Here is the patch after asking it to add the table to config.xml:
0001-feat-Add-customer-subscription-model-and-database-ta.patch

Edit: Adding "The install-1.0.0.php file should have been an upgrade." it correctly moved the install-1.0.0.php file to app/code/core/Mage/Customer/sql/customer_setup/upgrade-1.6.2.0.7-1.6.2.0.8.php and updated the <version> in config.xml as well. So it seems to have decent smarts when it comes to the "Magento Way".

[fballiano]

for magento, creating submodels/embedding, it's for sure the way to go

[sreichel]

2-3 years ago i tried to create a M1 module via ChatGTP.

First it made suggestions to create an M2-moudole, but after more detailed instructions and linking to SE, it was able to print correct moudule-structure.

It start with "please create an openmage module with vendor X and module name Y."

It may help, but at the moment any expierenced dev should be faster to create an extension. (n98-skeleton, module creator, ...). I dont believe it can create complex code right now. (e.g. admin config fields with dropdown fields, dependencies ...)

[colinmollenhour]

I've used Claude before to generate test cases for a different project with good success. I bet aider would be able to generate test cases for openmage pretty well without much work (adding enough context but not too much) . Especially the reasoning models should be good at this.

[sreichel]

Pinned https://github.com/Aider-AI/aider 👍

[sreichel]

jetbrains/phpstorm also offers AI support at $10/month. Maybe use it for generating unit tests?

https://blog.jetbrains.com/phpstorm/2024/02/ai-for-php-how-to-automate-unit-testing-using-ai-assistant/

Use funding for it (at least for 2-3 months?)

[kiatng]

I use cursor + Claude for a few months now. For me, it's amazing, may be I have low expectations. It's USD20/month, totally worth it. For OpenMage, it gets it right 50% of the time. For other, like new Python project, it provides 100% of the code, since I know next to nothing. It makes it much easier to experiment with AI tech such as LangChain.

[JesperHerrloff]

If you can code it is okay to use AI, but if you cannot understand the code it can open up security issues and hard to debug code.

I use PhpStorm with AI, but I only sometimes ask about feedback about some performance related stuff to get some other views of the issue :)

[pquerner]

I only have this enabled from within Phpstorm:

and it works nice enough. So far its getting pretty good at guessing what I am trying to accomplish.
Plus, Magicento helps out too.

I dont want my IDE to talk to an "AI" on the internet for code completion. I think thats wrong.

[colinmollenhour]

Just curious why you think it's wrong?

[pquerner]

Let me rephrase. I think its wrong to give AI tools access context to the projects source code. I dont know what the "AI" will do with that. Maybe it means part of that gets inserted back into their databases. Its not always my code and its not always my desire to have that possibly happen.

If you ask an "AI" without a projects source without context to the projects source I think its "fine", but 8 times out of 10 for me the "AI" just guessed something based on my prompts words. But it could'nt have worked when implemented.
Things like "how do I do XY in system.xml files in Magento" and then it tells you "ah you can add a <foo_bar> field that does that.
And then you search for foo_bar implementation in Magento source and its not there. The AI made it up. Based on some word from your prompt. Nice energy and time waste.

Of course if you have the power to run an AI LLM all locally and "learn" from your projects sources, that would be something else. But I didnt see that yet. If such things even exist. As in "get X, give it your source files, let it learn on its own". Without tinkering again.

[colinmollenhour]

Certainly, you should know the policies and trust the company that is offering the service, just like you have to trust Github if you use them to host your repo or Amazon if you use AWS. I think that is a real concern but easily solvable.

It's hard to measure how helpful they are... Aside from the sunk cost of learning the tools, there are times when it's like magic and you're really impressed, and then other times where it falls flat. I think they are at a point where they can help with simple tasks like scaffolding and rubber-ducking, but there is plenty they can't help with yet, too.

"learn" from your projects sources, that would be something else

That's what I'm saying it can do already. In Aider you use the .aiderignore file to tell it which sources it can and can't use, and it "embeds" them in the LLM. The context limit is probably the biggest challenge currently, but it is increasing with new LLMs and already to a point where it can be useful. It would be great if it could just take an entire large repo but for now we have to cull out the irrelevant stuff manually. Sounds like a good task for an AI... 🤣

[pquerner]

As I have first understood it, it was a "send your code always to AI provider". But you can also just send it to your locally running LLM. Whatever good that does, since normally we dont have such hardware available and thus "have to" a) rely on a much smaller model (and get weaker or no response at all), b) get response much much slower, c) or nothing at all.

Apart from your video you shared earlier, have you had success on "giving it access" to your local, private modules and the OpenMage core?

[colinmollenhour]

I use PhpStorm with GitHub Copilot daily, and I find the auto-completion to be helpful often enough that I prefer to use it, but that's online. It's basically the "enable and forget it" solution. When I tried locally with Qwen on a RTX 2060 the results were underwhelming.

I've not tried to set up Aider with a local LLM yet, but based on past experience, it's well worth $20 to just reserve some API capacity from Anthropic for Claude or similar since it's just so much faster and you can use the best models. If I had a better GPU (12+GB VRAM) I might try a local LLM again, but I'm ok with the terms for OpenAI and Anthropic in that they would be breaking the law to steal my code, and even if they did, I don't foresee being irreparably harmed by it, not that I would even know if they did...

[sreichel]

I'd not use it for generating core-code, but if it can automatically can create unit-test it would be a huge help. It does not need to be nearly perfect.

• add test classes
• add test methods for all public methods (even empty stubs would help)
• ...

[colinmollenhour]

It definitely works best when your code has comments. E.g. if you have a comment that says // Disable the shield and then some cryptic code that follows, you're far more likely to get back a testDisableShield result than without the comment.

[Flyingmana]

I had the luck to reveive a Training for Github Copilot recently, and also use it in a project of similar size of OpenMage.

So I can say, the size might be less of a problem. One Hint we got was, that copilot takes the strongest inspiration from currently opened files of the IDE, but also then it hallucinates a lot what I see.
For OpenMage the problem might be more the specific patterns being more unique, so the general training is not aligning so good.

But plain unitTests are generated well for independent classes. Or generating additional cases in dataProviders of Tests.

Something I would also add to your list, is the bare PHPstorm. The version of 2024 did make strong improvements on autocompleation/code generation even when not used with an AI companion. At least for the simple code constructs like loops.

[sreichel]

One Hint we got was, that copilot takes the strongest inspiration from currently opened files of the IDE

This works nearly perfect for unit tests.

"create unit test for Mage_Some_Class" similiar to this file" (90%)
"use generators for data-providers" (99%)
"apply coding-style" (99.9%)

... BUT copilot cant create files. So its not possible to automatically create the tests. :(

[colinmollenhour]

I've been living under a rock apparently, was told "Bruh" when I didn't know what "Cursor Rules" are.. So for anyone else not in the know: https://github.com/PatrickJS/awesome-cursorrules?tab=readme-ov-file
If I come up with a cursor rule file for OpenMage I'll let you know but don't expect it any time soon..

[colinmollenhour]

Who writes anything from scratch anymore? Here's one I cooked up with some AI help and manual tweaking.. I haven't tried it out yet, but now I'm really curious..

.cursorrules

I’m working on OpenMage, which is basically Magento 1 with ongoing community love. If I ask about something, think “How would this work in Magento 1?” first.

### Stick to Magento 1 Structure and Logic  
Assume I’m working with the OpenMage codebase, which mirrors Magento 1. That means `app/code/local`, `app/code/community`, and `app/code/core` (though I avoid touching core unless I'm hacking at core features). Keep things in line with Magento 1’s module structure—controllers, models, blocks, helpers, etc.

I use `layout.xml` files for frontend stuff— `<reference>`, `<block>`, `<action>`, all that jazz. Suggest solutions using XML layouts over hardcoded templates when possible.

If I need a new page or widget, lean toward `local.xml` or module-specific layout files rather than jumping to PHTML hacks.

### PHP Style: Magento 1 Flavor  
Respect Magento 1 era `Mage::getModel()`, `Mage::helper()`, and `Varien_Object` conventions. You can use strict typing and other fancy PHP 8 sytnax like arrow functions but don't go overboard.

If I need a collection, go for `Mage::getResourceModel()` rather than `Mage::getModel()`.

I’m used to Magento’s EAV system for products, categories, and customers. Suggest queries or model methods that fit that style.

If I need something custom, I’m okay with raw SQL via `$installer->run()` in setup scripts or `$connection->query()`. Just keep it Magento-friendly.

### No Magento 2 or Modern Framework Stuff  
Don’t suggest Magento 2 Dependency Injection, Symfony components, or anything that feels like a rewrite. I’m here for the Magento 1/OpenMage way—gritty and procedural.
If I ask for something “modern,” clarify with me first—I might just mean “works with PHP 8.4.”

### Templates and Frontend  
I like .phtml files with `<?php echo $this->getSomething(); ?>` —keep it simple and block-driven.
Assume I’m using the RWD theme or a custom theme built off base/default. No Tailwind or React unless I explicitly ask for it.

### Module Development  
When I build something, assume I want a custom module in `app/code/local/MyNamespace/MyModule`. Give me `etc/config.xml`, `etc/system.xml`, and maybe a `sql/` setup script if needed.

Stick to Magento 1 naming: `MyNamespace_MyModule_Model_Something`, not camelCase or PSR-12. For everything else I like PER 2.0.

### Wiki Awareness
- I’ve got a folder called `docs/content` in my project with Markdown files documenting OpenMage setup—modules, workflows, everything.
- When I ask questions, check those `.md` files first for context before guessing or pulling generic answers.
- If a file like `module-installation.md` or `custom-eav.md` has relevant info, use it and mention where you found it 

### Keep It Practical and SHORT
I don’t need long-winded theory—give me code snippets, file paths, and “put this here” advice. I’ll tweak it to fit.

If something’s unclear, ask me a quick question like “You mean on the frontend or admin?”—I’ll clarify fast.

That’s it! Help me crank out OpenMage code like it’s 2015, and we’ll get along great.

[kiatng]

I didn't know cursor rules. Awesome.