Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add dp_session_query function #8

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

dungmv56
Copy link

@dungmv56 dungmv56 commented Apr 8, 2021

Currently, dataplane support a limited info session for pipeline plugins. This adding support query session information from pipeline plugin.

Example:

struct dp_session_info info;
dp_session_query(s, SESSION_ATTR_ALL, info);

Additional, this pull request was made to support functionality of flowstat plugin.
vyatta-flowstat-plugin
vyatta-service-flowstat

@nickbroon nickbroon requested a review from dfawcus April 8, 2021 14:01
@nickbroon
Copy link
Contributor

Thanks for splitting this into a PR with the dataplane enhancements and moving the plugin and control plane daemon to their own repos/packages.

@nickbroon
Copy link
Contributor

Related Jira ticket: https://danosproject.atlassian.net/browse/DAN-147

@dfawcus
Copy link
Contributor

dfawcus commented Apr 8, 2021

Please bear with us in reviewing this, some of us need to refresh our memories on the dp session infra and its mechanisms.

On the face of it, you're making use of a facility which the original author added, but to date has not been used - the session watch mechanism. Since the code mentions there can only be one watcher per type, we'll need to review the original design docs for why that mechanism was added in the first place.

I suspect @pjaitken and/or @gavin-shr should also be looking at these three PRs.

@dungmv56 dungmv56 force-pushed the dp_query branch 2 times, most recently from 9716106 to 7d537f0 Compare April 12, 2021 05:01
@nickbroon
Copy link
Contributor

nickbroon commented Apr 13, 2021

@dungmv56 Are you also interesting in migrating your https://github.com/dungmv56/vyatta-service-flowstat and https://github.com/dungmv56/vyatta-flowstat-plugin repositories into the github.com/danos project and having the feature included as part of the default DANOS community image?

return -1;

info->se_id = s->se_id;

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If info was populated with all available information, then query wouldn't be necessary which would make it easier to add new information in future.

/**
* Session attribute.
*/
enum dp_session_attr {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the need to request specific attrs due to the plugin directly outputting the returned JSON?
The plugin should retrieve the necessary attrs from the JSON and format them into a suitable table.
Therefore it should not matter if additional attrs are returned.
The dataplane could return all of the attrs, and the plugin only output the relevant values.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The plugin registers a session_watch to hook when session state or stats changed. This was called from dataplane forwarding path and I try to keep plugin not affect forwarding performance as much as possible.

I see that returning all attrs from dataplane make easier for adding new attributes but I afraid it could affect performance. That why I added a query as parameter.

const char *se_app_type;

// misc
time_t timestamp;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please use tabs in lines 106-118 so the indentation aligns with the rest of the struct.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 78055df

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add your copyright message to the modified files.

Fixed in 278e9bf

@pjaitken
Copy link
Contributor

Please add your copyright message to the modified files.

@dungmv56
Copy link
Author

@dungmv56 Are you also interesting in migrating your https://github.com/dungmv56/vyatta-service-flowstat and https://github.com/dungmv56/vyatta-flowstat-plugin repositories into the github.com/danos project and having the feature included as part of the default DANOS community image?

Yes, very happy if it was a part of DANOS community image.

@nickbroon
Copy link
Contributor

Please squash the fix commits into the original commit.

@nickbroon
Copy link
Contributor

nickbroon commented Apr 14, 2021

@dungmv56 Are you also interesting in migrating your https://github.com/dungmv56/vyatta-service-flowstat and https://github.com/dungmv56/vyatta-flowstat-plugin repositories into the github.com/danos project and having the feature included as part of the default DANOS community image?

Yes, very happy if it was a part of DANOS community image.

That's great to hear. Once these changes to the dataplane are reviewed, and if merged, we can then look at bringing dungmv56/vyatta-service-flowstat and dungmv56/vyatta-flowstat-plugin into DANOS. This will likely consist of creating empty danos/vyatta-service-flowstat and danos/vyatta-flowstat-plugin repos and inviting you to submit PRs with the code so that appropriate code/data-model review can take place.

(Couple of initial comments after a quick look at those repos. It would be good to extend the readme in the vyatta-service-flowstat repo with those parts from the readme in vyatta-flowstat-plugin that are concerned with config/op commands. Contributions to the DANOS project need to use the Apache License 2.0. More detailed review of the code and data-model will take place on initial submission by others)


Follow-up: I think calling the repos and source packages 'dataplane-flowstat-plugin' and 'danos-service-flowstat' would be appropriate, to avoid re-using the 'vyatta' name.

@nickbroon
Copy link
Contributor

nickbroon commented Apr 14, 2021

danos/dataplane-flowstat-plugin and danos/danos-service-flowstat have been created. Please submit PRs with your code.

You'll need to update your debian package names to reflect these new repo names.

@nickbroon
Copy link
Contributor

For the dataplane plugin repos, lintian reports some errors and warnings:

[   79s] + lintian -i /usr/src/packages/flowstat_3.12.42+1.1_amd64.changes
[   80s] W: flowstat source: section-area-mismatch Package vyatta-dataplane-flowstat-plugin
[   80s] N: 
[   80s] N:    The debian/control file places the indicated binary package in a
[   80s] N:    different archive area (main, contrib, non-free) than its source package
[   80s] N:    or other binary packages built from the same source package. The source
[   80s] N:    package and any binary packages it builds must be in the same area of
[   80s] N:    the archive, with the single exception that source packages in main may
[   80s] N:    also build binary packages in contrib if they build binary packages in
[   80s] N:    main.
[   80s] N:    
[   80s] N:    Severity: normal, Certainty: certain
[   80s] N:    
[   80s] N:    Check: control-file, Type: source
[   80s] N: 
[   80s] W: flowstat source: section-area-mismatch Package libvyatta-dataplane-flowstat-plugin-proto-support
[   80s] E: flowstat source: invalid-standards-version 3.11.72
[   80s] N: 
[   80s] N:    The source package refers to a Standards-Version which never existed.
[   80s] N:    Please update your package to latest Policy and set this control field
[   80s] N:    appropriately.
[   80s] N:    
[   80s] N:    Severity: important, Certainty: certain
[   80s] N:    
[   80s] N:    Check: standards-version, Type: source
[   80s] N: 

@nickbroon
Copy link
Contributor

The control plane daemon and data-model repo lintian reports some errors and warnings:

[   59s] + lintian -i /usr/src/packages/vyatta-service-flowstat_3.12.42+1.1_amd64.changes
[   60s] E: vyatta-service-flowstat source: missing-separator-between-items in vyatta-service-flowstat-v1-yang depends field between '${misc:Depends}' and '${perl:Depends}'
[   60s] N: 
[   60s] N:    The given field in the debian/control file contains a list of items
[   60s] N:    separated by commas and pipes. It appears a separator was missed between
[   60s] N:    two items. This can lead to bogus or incomplete dependencies, conflicts
[   60s] N:    etc.
[   60s] N:    
[   60s] N:    Severity: important, Certainty: certain
[   60s] N:    
[   60s] N:    Check: control-file, Type: source
[   60s] N: 
[   60s] W: vyatta-service-flowstat source: section-area-mismatch Package vyatta-flowstatd
[   60s] N: 
[   60s] N:    The debian/control file places the indicated binary package in a
[   60s] N:    different archive area (main, contrib, non-free) than its source package
[   60s] N:    or other binary packages built from the same source package. The source
[   60s] N:    package and any binary packages it builds must be in the same area of
[   60s] N:    the archive, with the single exception that source packages in main may
[   60s] N:    also build binary packages in contrib if they build binary packages in
[   60s] N:    main.
[   60s] N:    
[   60s] N:    Severity: normal, Certainty: certain
[   60s] N:    
[   60s] N:    Check: control-file, Type: source
[   60s] N: 
[   60s] W: vyatta-service-flowstat source: section-area-mismatch Package vyatta-service-flowstat-v1-yang
[   60s] W: vyatta-service-flowstat source: section-area-mismatch
[   60s] W: vyatta-service-flowstat source: debhelper-but-no-misc-depends vyatta-service-flowstat-v1-yang
[   60s] N: 
[   60s] N:    The source package uses debhelper, but it does not include
[   60s] N:    ${misc:Depends} in the given binary package's debian/control entry. Any
[   60s] N:    debhelper command may add dependencies to ${misc:Depends} that are
[   60s] N:    required for the work that it does, so recommended best practice is to
[   60s] N:    always add ${misc:Depends} to the dependencies of each binary package if
[   60s] N:    debhelper is in use.
[   60s] N:    
[   60s] N:    Refer to the debhelper(7) manual page for details.
[   60s] N:    
[   60s] N:    Severity: normal, Certainty: possible
[   60s] N:    
[   60s] N:    Check: debhelper, Type: source
[   60s] N: 
[   60s] E: vyatta-service-flowstat source: missing-build-dependency-for-dh-addon python3 => dh-python
[   60s] N: 
[   60s] N:    The source package appears to be using a dh addon but doesn't build
[   60s] N:    depend on the package that actually provides it. If it uses it, it must
[   60s] N:    build depend on it.
[   60s] N:    
[   60s] N:    Severity: important, Certainty: possible
[   60s] N:    
[   60s] N:    Check: debhelper, Type: source
[   60s] N: 
[   61s] W: vyatta-service-flowstat source: missing-license-paragraph-in-dep5-copyright gpl-2.0-only (paragraph at line 38)
[   61s] N: 
[   61s] N:    The files paragraph in the machine readable copyright file references a
[   61s] N:    license, for which no standalone license paragraph exists.
[   61s] N:    
[   61s] N:    Refer to
[   61s] N:    https://www.debian.org/doc/packaging-manuals/copyright-format/1.0/ for
[   61s] N:    details.
[   61s] N:    
[   61s] N:    Severity: normal, Certainty: possible
[   61s] N:    
[   61s] N:    Check: source-copyright, Type: source
[   61s] N: 
[   61s] E: vyatta-service-flowstat source: invalid-standards-version 3.12.42
[   61s] N: 
[   61s] N:    The source package refers to a Standards-Version which never existed.
[   61s] N:    Please update your package to latest Policy and set this control field
[   61s] N:    appropriately.
[   61s] N:    
[   61s] N:    Severity: important, Certainty: certain
[   61s] N:    
[   61s] N:    Check: standards-version, Type: source
[   61s] N: 
[   61s] E: vyatta-flowstatd: init.d-script-not-included-in-package etc/init.d/vyatta-flowstatd
[   61s] N: 
[   61s] N:    The /etc/init.d script is registered in the postinst script, but is not
[   61s] N:    included in the package.
[   61s] N:    
[   61s] N:    Severity: important, Certainty: certain
[   61s] N:    
[   61s] N:    Check: init.d, Type: binary
[   61s] N: 

@dungmv56
Copy link
Author

Contributions to the DANOS project need to use the Apache License 2.0. More detailed review of the code and data-model will take place on initial submission by others)

I have a question about the license. Currently two repos use some modified code from DANOS which has some licenses like LGPL, GPL, BSD. And that result to that repos will be licensed in LGPL/GPL. So, is it allow to license repos as LGPL and make a pull request?

@nickbroon
Copy link
Contributor

Contributions to the DANOS project need to use the Apache License 2.0. More detailed review of the code and data-model will take place on initial submission by others)

I have a question about the license. Currently two repos use some modified code from DANOS which has some licenses like LGPL, GPL, BSD. And that result to that repos will be licensed in LGPL/GPL. So, is it allow to license repos as LGPL and make a pull request?

Yes, if existing LGPL/GPL/BSD code is being copied/modified then that license continues to apply. It's worth mentioning that in the PR. Only new code need use the Apache 2 license.

@dfawcus
Copy link
Contributor

dfawcus commented Apr 15, 2021

I have a question about the license. Currently two repos use some modified code from DANOS which has some licenses like LGPL, GPL, BSD. And that result to that repos will be licensed in LGPL/GPL. So, is it allow to license repos as LGPL and make a pull request?

Yes, if existing LGPL/GPL/BSD code is being copied/modified then that license continues to apply. It's worth mentioning that in the PR. Only new code need use the Apache 2 license.

In which case, is it worth splitting the stuff with difference license conditions in to distinct files?
Is LTO used for building the plugins? If it was, it could mitigate any detrimental effect from the split.

@dungmv56
Copy link
Author

dungmv56 commented Apr 15, 2021

In which case, is it worth splitting the stuff with difference license conditions in to distinct files?
Is LTO used for building the plugins? If it was, it could mitigate any detrimental effect from the split.

Yes, when building plugin debian package, it used LTO.

@nickbroon
Copy link
Contributor

nickbroon commented Apr 15, 2021

These together form the full feature:
danos/dataplane-flowstat-plugin#1
danos/danos-service-flowstat#1

@nickbroon nickbroon requested a review from gavin-shr April 15, 2021 17:11
Add support query session information from pipeline plugin.

Signed-off-by: Dung Man <[email protected]>
nickbroon referenced this pull request in nickbroon/vyatta-dataplane Apr 28, 2021
se_sen field in struct session may be NULL in two cases for newly
created sessions not yet added to sentry hash list, or after sesssion
removed from sentry hash list during reclaim in session gc.

This causes the following segmentaion fault infrequently.

[Current thread is 1 (Thread 0x7ff5c3fff700 (LWP 30235))]
 #0  0x000055ce31a8978d in csync_get_session_from_init_sentry (
    cse=<synthetic pointer>, cs=<synthetic pointer>, sp=0x7ff5f004ef36)
    at ../src/npf/csync/csync_session_unpack.c:38
 #1  csync_session_unpack_update (csu=0x7ff5f004ef2e)
    at ../src/npf/csync/csync_session_unpack.c:71
 #2  csync_unpack_session (size=<optimized out>, msg=0x7ff5f004ef26)
    at ../src/npf/csync/csync_session_unpack.c:421
 #3  csync_recv_session_update (frame=<optimized out>)
    at ../src/npf/csync/csync_session_unpack.c:501
 #4  0x000055ce31b2d57d in csync_restore_sessions (n=<optimized out>,
    flist=<optimized out>) at ../src/csync/csync_transfer.c:218
 #5  csync_pull_batch (info=0x7ff58400b880) at
../src/csync/csync_transfer.c:506
 #6  csync_xfer_backup (pipe=0x7ff58400b8e0, arg=0x7ff58400b880)
    at ../src/csync/csync_transfer.c:301
 #7  0x00007ff6629618d3 in ?? () from
/usr/lib/x86_64-linux-gnu/libczmq.so.4
 #8  0x00007ff6611ad4a4 in start_thread ()
   from /lib/x86_64-linux-gnu/libpthread.so.0
 danos#9  0x00007ff660eefd0f in clone () from /lib/x86_64-linux-gnu/libc.so.6

Fixed by doing a safe derefernce and checking for NULL.

VRVDR-54586
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants