Commit 5cbaf196 authored by Dmitry Ganyushin's avatar Dmitry Ganyushin
Browse files

Merge branch 'master' into http-connector 

* master:
  Update readme for heat transfer example with new location and build instructions
  Ignore tests with defects for now
  Adapt libfabric dataplane of SST to Cray CXI provider (#3672)
  ci: fix path to lsan suppressions, fix broken gh status post
  Use adios2_mode_readRandomAccess in matlab open to make it work for BP5 (#3956)
  Add Global Array Capabilities and Limitations
  Add Section for Anatomy of an ADIOS Program
  Enable Shell-Check for gh-actions scripts
  Enable Shell-Check for circle CI scripts
  Enable Shell-Check for tau contract scripts
  Enable Shell-Check for scorpio contract scripts
  Enable Shell-Check for lammps contract scripts
  Delete VTK code in examples
  Fix MATLAB bindings for MacOS (#3950)
  Set the compiler for the Kokkos DataMan example to what is used to build Kokkos
  Fix the HIP architecture CMAKE variable (#3931)
  perfstubs 2023-11-27 (845d0702) (#3944)
  Revert "Only rank 0 should print the initialization message in perfstub"
  CI Contract: Build examples with external ADIOS
  Example using DataMan with Kokkos buffers
  Propagating the GPU logic inside the DataMan engine
  ci: Use mpich built with ch3:sock:tp for faster tests
  ReadMe.md: Mention 2.9.2 release
  Cleanup server output a bit (#3914)
  ci: set openmpi and openmp params
  Example using Kokkos buffers with SST
  Changes to MallocV to take into consideration the memory space of a variable
  Change install directory of Gray scott files again
  ci,crusher: increase supported num branches
  ci: add shellcheck coverage to source and testing
  Change install directory of Gray scott files
  Only rank 0 should print the initialization message in perfstub
  Defining and computing derived variables (#3816)
  Add Remote "-status" command to see if a server is running and where (#3911)
  examples,hip: use find_package(hip) once in proj
  Add Steps Tutorial
  Add Operators Tutorial
  Add Attributes Tutorial
  Add Variables Tutorial
  Add Hello World Tutorial
  Add Tutorials' Download and Build section
  Add Tutorials' Overview section
  Improve bpStepsWriteRead* examples
  Rename bpSZ to bpOperatorSZWriter
  Convert bpAttributeWriter to bpAttributeWriteRead
  Improve bpWriter/bpReader examples
  Close file after reading for hello-world.py
  Fix names of functions in engine
  Fix formatting warnings
  Add dataspaces.rst in the list of engines
  Add query.rst
  cmake: find threads package first
  docs: update new_release.md
  Bump version to v2.9.2
  ci: update number of task for mpich build
  clang-format: Correct format to old style
  Merge pull request #3878 from anagainaru/test-null-blocks
  Merge pull request #3588 from vicentebolea/fix-mpi-dp
  bp5: make RecMap an static anon namespaced var
  Replace LookupWriterRec's linear search on RecList with an unordered_map. For 250k variables, time goes from 21sec to ~1sec in WSL. The order of entries in RecList was not necessary for the serializer to work correctly. (#3877)
  Fix data length calculation for hash (#3875)
  Merge pull request #3823 from eisenhauer/SstMemSel
  gha,ci: update checkout to v4
  Blosc2 USE ON: Fix Module Fallback
  cmake: correct prefer_shared_blosc behavior
  cmake: correct info.h installation path
  ci: disable MGARD static build
  operators: fix module library
  ci: add downloads readthedocs
  cmake: Add Blosc2 2.10.1 compatibility.
  Fix destdir install test (#3850)
  cmake: update minimum cmake to 3.12 (#3849)
  MPI: add timeout for conf test for MPI_DP (#3848)
  MPI_DP: do not call MPI_Init (#3847)
  install: export adios2 device variables (#3819)
  Merge pull request #3799 from vicentebolea/support-new-yaml-cpp
  Merge pull request #3737 from vicentebolea/fix-evpath-plugins-path
  Partial FFS Upstream, only changes to type_id
  bpls -l  with scalar string variable: print the value (since min/max is empty). This changes the code for all types using Engine.Get() to get the value now.
  Set AWS version requirement to 1.10.15 and also turn it OFF by default as it is not a stable feature of ADIOS just yet.
  Fix local values block reading
  docs,ci: backport fixes for readthedocs
parents a451cf11 68c23fcb

.circleci/config.yml

+4 −0
Original line number Diff line number Diff line
@@ -2,6 +2,10 @@ defaults: &defaults 
  steps:

    - checkout:

        path: source

    - run:

        # https://discuss.circleci.com/t/circle-working-directory-doesnt-expand/17007

        name: "Fix CIRCLE_WORKING_DIRECTORY"

        command: echo 'CIRCLE_WORKING_DIRECTORY="${CIRCLE_WORKING_DIRECTORY/#\~/$HOME}"' >> $BASH_ENV

    - run:

        name: CDash

        command: bash source/scripts/ci/circle/postCDashStatus.sh

.github/ISSUE_TEMPLATE/new_release.md

+4 −1
Original line number Diff line number Diff line
@@ -33,7 +33,7 @@ git push 
- [ ] Create PR (BASE to master if release_@MAJOR@@MINOR@ does not exists; otherwise release_@MAJOR@@MINOR@)

- [ ] Ask for review

- [ ] Merge PR

- [ ] Create Tag commit `git tag -a v@VERSION@ the_merge_commit`

- [ ] Create Tag commit `git tag -a -m 'v@VERSION' v@VERSION@ the_merge_commit`

- [ ] Create Release in GitHub page

  - Use the following script for getting the PR of this release

    - `./scripts/developer/create-changelog.sh v@VERSION@ v@OLD_RELEASE@`
@@ -66,5 +66,8 @@ git push origin master 
  - CondaForge robot should do this for you automatically, expect a new PR at

    https://github.com/conda-forge/adios2-feedstock a couple of hours after the

    release.

- [ ] Submit a MR for ParaView Superbuild to use v@VERSION@ release.

- [ ] Update the website to point to the v@VERSION@ release

- [ ] Write an announcement in the ADIOS-ECP mail-list

  (https://groups.google.com/a/kitware.com/g/adios-ecp)

.github/workflows/everything.yml

+10 −6
Original line number Diff line number Diff line
@@ -7,7 +7,7 @@ 
# Note the use of multiple checkout actions in most jobs.  This has been

# implemented to allow the use of CI scripts at a different ref or sha than

# the source code they're evaluating.  For push events (when a pull_request is

# merged) ther is no difference.  However, for pull_request events this allows

# merged) there is no difference.  However, for pull_request events this allows

# us test code at the head of a pull_request using the CI scripts from the

# prospectively merged pull_request, which will include any CI updates that

# may have made it to the target branch after the pull_request was started.
@@ -127,14 +127,17 @@ jobs: 
        os: [ubuntu20.04]

        compiler: [gcc8, gcc9, gcc10, gcc11, clang6, clang10]

        shared: [shared]

        parallel: [ompi]

        parallel: [mpich]

        include:

          - os: ubuntu20.04

            compiler: gcc10

            parallel: mpich

            compiler: gcc8

            parallel: ompi

          - os: ubuntu20.04

            compiler: gcc8

            parallel: serial

          - os: ubuntu20.04

            compiler: clang6

            parallel: ompi

          - os: ubuntu20.04

            compiler: clang6

            parallel: serial
@@ -215,7 +218,7 @@ jobs: 
      matrix:

        os: [el8]

        compiler: [icc, oneapi]

        parallel: [ompi]

        parallel: [mpich]



    steps:

      - uses: actions/checkout@v4
@@ -448,8 +451,9 @@ jobs: 
    strategy:

      fail-fast: false

      matrix:

        code: [lammps, tau]

        code: [examples, lammps, tau]

        include:

          - code: examples

          - code: lammps

            repo: pnorbert/lammps

            ref: fix-deprecated-adios-init

.gitlab/config/generate_pipelines.py

+51 −11
Original line number Diff line number Diff line
@@ -18,9 +18,43 @@ import urllib3 
urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)





def request_as_dict(url):

    r = requests.get(url + '?per_page=100', verify=False)

    return r.json()

class skip_after_n_successes:

    def __init__(self, default_value, n):

        self.runs_max = n

        self.runs_current = 0

        self.default_value = default_value



    def __call__(self, fn, *args, **kwargs):

        if self.runs_current >= self.runs_max:

            return self.default_value



        ret = fn(*args, **kwargs)

        if ret:

            self.runs_current += 1

        return ret





def http_get_request(*args, **kwargs):

    kwargs['verify'] = False

    return requests.get(*args, **kwargs)





def request_as_list(url, *args, **kwargs):

    current_url = url

    body_json = []

    while current_url:

        response = http_get_request(current_url, *args, **kwargs)

        body_json += response.json()



        header = response.headers

        current_url = None

        if 'link' in header:

            links = re.search(

                r'(?<=\<)([\S]*)(?=>; rel="next")', header['link'], flags=re.IGNORECASE)

            if links is not None:

                current_url = links.group(0)



    return body_json





def add_timestamp(branch):
@@ -44,7 +78,12 @@ def has_no_status(branch): 
        gh_commit_sha = branch['commit']['parent_ids'][1]



    # Query GitHub for the status of this commit

    commit = request_as_dict(gh_url + '/commits/' + gh_commit_sha + '/status')

    response = http_get_request(

        gh_url + '/commits/' + gh_commit_sha + '/status')

    if int(response.headers['x-ratelimit-remaining']) <= 0:

        raise ConnectionError(response.json())



    commit = response.json()

    if commit is None or 'sha' not in commit:

        return False
@@ -88,14 +127,15 @@ gh_url = 'https://api.github.com/repos/' + args.gh_name 
with open(args.template_file, 'r') as fd:

    template_str = fd.read()



    branches = request_as_dict(gl_url + '/repository/branches')

    branches = map(add_timestamp, branches)

    branches = filter(is_recent, branches)

    branches = filter(has_no_status, branches)



    # Select the arg.max most least recent branches

    branches = request_as_list(gl_url + '/repository/branches')

    branches = [add_timestamp(branch) for branch in branches]

    branches = [b for b in branches if is_recent(b)]

    branches = sorted(branches, key=lambda x: x['dt'])

    branches = itertools.islice(branches, args.max)



    # Skip running (and return true) has_no_status after returning True args.max times.

    # We need this not to hog the Github Rest API draconian ratelimit.

    run_n_times = skip_after_n_successes(default_value=False, n=args.max)

    branches = [b for b in branches if run_n_times(has_no_status, b)]



    for branch in branches:

        print(template_str.format(

.shellcheck_exclude_paths

+0 −6
Original line number Diff line number Diff line 
scripts/ci/circle/postCDashStatus.sh

scripts/ci/circle/run.sh

scripts/ci/gh-actions/check-branch-name.sh

scripts/ci/gh-actions/get-changed-files.sh

scripts/ci/gh-actions/macos-setup.sh

scripts/ci/gh-actions/run.sh

scripts/ci/scripts/github-prs-to-gitlab.sh

scripts/ci/scripts/run-clang-format.sh

scripts/ci/scripts/run-flake8.sh