Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement Multithreading #1610

Open
wants to merge 29 commits into
base: master
Choose a base branch
from
Open

Implement Multithreading #1610

wants to merge 29 commits into from

Conversation

valentinbreiz
Copy link
Member

@valentinbreiz valentinbreiz commented Dec 27, 2020

This pull request is not to merge in master yet, this is more an opening for discussion on the implementation of multithreading in Cosmos.

All the original code of the first commit comes from @Og-Rok's work from Aura's fork of Cosmos. (the original pull request can be found here if needed: https://github.com/aura-systems/Cosmos/pull/40)

Here is a short demonstration video: https://www.youtube.com/watch?v=rea5sAkYyec

To do:

  • fix PC Speaker and process scheduler conflict due to PIT
  • alloc from new heap system
  • add test kernel

ProcessContext.m_CurrentContext = context;

IOPort counter0 = new IOPort(0x40);
IOPort cmd = new IOPort(0x43);
Copy link
Member Author

@valentinbreiz valentinbreiz Dec 27, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Find a way to use another port or something else than Pit (APIC timer?)
With this using PC Speaker won't be possible and will hang the processor scheduler due to PIT conflicts

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe use PIT implemented in Cosmos, not IOPort PIT?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@AsertCreator they are one in the same

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe you can implement the nanosecond time in RTC, and use RTC to approximate? or maybe (pure speculation, as a junior dev who knows nothing about low level) reimplement PIT wait to tally up the number of times IRQ 0 has been triggered since the os was powered on, and use this to wait x amount of time. in the case of the second option, it would be possible to implement a system where multiple things can use PIT wait at the same time. see osdev wiki.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

RTC is too slow for scheduling and idk if we could make interrupts with that

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In fact RTC may not be a bad idea until we have HPET or local APIC timers, it indeed can generate IRQs. I'll try, I hope this won't mess up date time

Copy link
Member Author

@valentinbreiz valentinbreiz Jan 17, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

01:35:10.769067         Msg: Text from kernel: SwitchTask
01:35:10.787021         Msg: Text from kernel: SwitchTask
01:35:10.802028         Msg: Text from kernel: SwitchTask
01:35:10.818118         Msg: Text from kernel: SwitchTask
01:35:10.834006         Msg: Text from kernel: SwitchTask
01:35:10.849588         Msg: Text from kernel: SwitchTask
01:35:10.865546         Msg: Text from kernel: SwitchTask

I can get no more than 60Hz but it's possible

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does that function as a stopgap until APIC gets finished?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh. I just looked at the commit and it appears y'all got the APIC timer working. Ok then.

@madelynwith5ns
Copy link

This would be very beneficial.

@Nik300
Copy link
Contributor

Nik300 commented Jan 3, 2021

Guys have a huge problem with while loops...
this is the code I am threading... and it just stops after some seconds

public void TaskRun()
{
            while (true)
                Sys.Emulation.Threading.TaskManager.Next();
}
public static void Next()
{
    foreach (Thread thread in LoadedThreads)
    {
       DateTime startTime = DateTime.Now;
       DateTime lastTime = startTime;
       while (lastTime.Second < startTime.Second + 1)
       {
            thread.exec.NextInstruction();
            lastTime = DateTime.Now;
        }
    }
}

Copy link
Contributor

@Nik300 Nik300 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really impressive changes here... but have some serious troubles with while loops!
I'll try to track down the error then publish my results

@Nik300
Copy link
Contributor

Nik300 commented Jan 4, 2021

UPDATE:
Ok so I just found out that the problem isn't the while loop but the keyboard instead...
Here's the thing:
When I load a task as long as I don't press any key it works... but as soon as I click a key the entire system shuts down

🤔

I'll dig more into that

EDIT: I think i may know the problem

The Console.ReadLine is being split across the threads needing it...
let's say that i have 'Thread1' and 'Thread2' and both use 'Console.ReadLine()':
the first thread starts and uses the keyboard to read the first 2 chars, then passes control to the second thread that takes the next two chars, and this happens until the user clicks the ENTER key and one of the two processes handles the result from the keyboard...

Now let's imagine a scenario where one of the two threads doesn't use 'Console.ReadLine()':
as soon as thread1 passes control to thread2 the system crashes, because the thread finds something in its stack that shouldn't be there!

The only way to fix this is by modifying the behaviour of Console.ReadLine

EDIT:
here's the proof:
Untitled

@charlesbetros
Copy link
Member

charlesbetros commented Jan 4, 2021 via email

@Og-Rok
Copy link
Contributor

Og-Rok commented Jan 4, 2021

From what i remember with this, a lot of the core kernel systems (interrupts, heap, device management) are not thread safe, meaning that you will get a lot of situations like this where threads are tripping each other up. A fix for things like the heap would be to implement something like spinlocks

@Nik300
Copy link
Contributor

Nik300 commented Jan 4, 2021

Is it any interrupt or just keyboard?

On Jan 4, 2021, at 12:21 PM, F4lc0131 @.***> wrote:  UPDATE: Ok so I just found out that the problem isn't the while loop but the keyboard instead... Here's the thing: When I load a task as long as I don't press any key it works... but as soon as I click a key the entire system shuts down 🤔 — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or unsubscribe.

as of now just the keyboard... I'll try to bust and fix those errors before releasing this to real cosmos

@Nik300
Copy link
Contributor

Nik300 commented Jan 4, 2021

Actually guys I had an idea on how we could handle this but I'll have to work on a "lock" system, so that a thread can acquire the lock over something (in this case the Console.ReadLine) tell me what you guys think (especially @valentinbreiz)

@Nik300
Copy link
Contributor

Nik300 commented Jan 4, 2021

Screenshot 2021-01-04 225222
For now this should do the job... but i'd like to do something more flexible

UPDATE:
Screenshot 2021-01-04 233606
Good news: it doesn't crash!
Bad news: after flushing the first thread, the input just hangs... 😢
have to work on this

UPDATE:
Screenshot 2021-01-05 000253
Got it working guys! I'm gonna pull request the code to @valentinbreiz 's fork so that he can port it to this pull request

@Nik300
Copy link
Contributor

Nik300 commented Jan 4, 2021

UPDATE:
Lol I just realized that there's something weirder going on...
ok so with the Console.ReadLine() fix I still have issues with loading more than one thread that use a while loop...
have to track that down too and fix that :(

UPDATE:
No, i was just being an idiot :D, just forgot to fix the main problem... going to compile and see if it works, but i'll go to bed either if it does/doesn't work

EDIT:
Nope doesn't work, ok i'll work on that tomorrow (i want to point out that as long as the second thread uses Console.ReadLine too, it works... just crashes when the second thread doesn't use it)

UPDATE:
Ok so I think I got another problem like that, and surprisingly the problem is with Console.WriteLine().

Sys.Thread(() => { while (true) Console.WriteLine("test"); });

that's the thread, and happens the same exact thing... shell I add a locksystem to WriteLine too? Or just use the same lock in common with ReadLine?? lemme know

@Nik300
Copy link
Contributor

Nik300 commented Jan 5, 2021

UPDATE:
2021-01-05 13-38-32
Fixed! I think that Cosmos.Clear() needs the same treatment... because as soon as I use the command "cls" the system shuts down

@madelynwith5ns
Copy link

With the trend we're seeing with methods like Console.ReadLine() Console.WriteLine() and Console.Clear() I'm guessing that there is other methods with this issue.

@valentinbreiz
Copy link
Member Author

valentinbreiz commented Jan 6, 2021

You're right @TFTWPhoenix, some parts of Cosmos are not thread safe and maybe needs some locks too
But WriteLine or Clear are working fine

@github-actions
Copy link

Stale pull request message

@ThomasBeHappy
Copy link
Contributor

Will this ever be finished? Kinda really waiting for this.

@selkij
Copy link
Contributor

selkij commented May 29, 2022

Any news?

@AsertCreator
Copy link
Contributor

i got this to work (maybe). currently there are two problems: stack corruption detection and Thread.Sleep method. first one thinks that stack was corrupted, although it just got replaced. second one is not implemented and makes thread sleep indefinitely, in other words, it makes thread die. when these are not used, multithreading somehow and somewhat works. do i need to submit PR?

@zarlo
Copy link
Member

zarlo commented Mar 20, 2023

i would make a PR as the owner of the RP does not have time to work on this

@AsertCreator
Copy link
Contributor

ok

Neil-Ciciglycerin

This comment was marked as outdated.


public void Lock()
{
while (gate != 0) { }
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should yield here so we dont waste as much cpu time

@Neil-Ciciglycerin
Copy link

Neil-Ciciglycerin commented Jan 17, 2024

i found a bug with how multithreading interacts with the AC97 driver here

@Neil-Ciciglycerin
Copy link

Neil-Ciciglycerin commented Jan 23, 2024

i found a bug with graphics. i have some system init code in a function called __sysstartup__. there's a null reference exception when init-ing canvas, on main thread or other thread.

@valentinbreiz
Copy link
Member Author

i found a bug with graphics. i have some system init code in a function called __sysstartup__. there's a null reference exception when init-ing canvas, on main thread or other thread.

Hello @Neil-Ciciglycerin I think this issue is also in the main branch unfortunally. Could you try and tell me please?

@zarlo zarlo enabled auto-merge January 23, 2024 20:04
/// <summary>
/// MMIOBase abstract class.
/// </summary>
public unsafe abstract class MMIOBase
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How does this differ from a MemoryBlock?

}
}
}

/*
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need this commented out code?

@@ -20,6 +21,8 @@ public enum ObjectGCStatus : ushort
public static unsafe class Heap
{
private static uint* StackStart;
private static Mutex mMemeoryGate = new Mutex();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typo

@@ -44,6 +47,11 @@ public static unsafe void Init()
/// <returns>New pointer with specified size while maintaining old data.</returns>
public static byte* Realloc(byte* aPtr, uint newSize)
{
if (mMemeoryGate != null)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How can the mMemoryGate ever be null? Seems like we are doing useless checks

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we could replace it with mMemeoryGate?.Lock(); it would do that same

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would prefer if we dont do needless checks to reduce the performance cost, as these calls happen quite often

}

[PlugMethod(PlugRequired = true)]
public static uint GetPointer(Object aVal) { return 0; }
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We already have

public static unsafe uint* GetPointer(object aObj) => throw null; // this is plugged
which does more or less the same?
If you need a uint we also have
public static unsafe uint GetSafePointer(object aObj)


else
{
new LiteralAssemblerCode("pushad");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not use the XS equivalent commands?

[Plug(Target = typeof(ObjUtilities))]
public static unsafe class ObjUtilitiesImpl
{
[PlugMethod(Assembler = typeof(ObjUtilitiesGetPointer))]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See previous comment if we really need this method new here.


public static void Ctor(ThreadStart aThis, ThreadStart aEntry)
{
Console.WriteLine("Thread started");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should be removed.

Console.WriteLine("Thread started");
}

// public static void SleepInternal(int ms)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this code be removed?


Console.WriteLine("Starting ACPI");
debugger.Send("ACPI Init");
ACPI.Start();

Console.WriteLine("Finding PCI Devices");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we should write as much to the console, since the user cant control this behavior.

@Neil-Ciciglycerin
Copy link

Neil-Ciciglycerin commented Jan 24, 2024

The last time I checked it wasn't in the main branch, although I was using userkit. I'll check now

@Neil-Ciciglycerin
Copy link

Yeah, it's in the main branch. I'll go start an issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.