Skip to content

Commit

Permalink
Remove FilterRowsTransformer & related code (#524)
Browse files Browse the repository at this point in the history
  • Loading branch information
stloyd authored Oct 5, 2023
1 parent 52f0e67 commit 61b2a6c
Show file tree
Hide file tree
Showing 16 changed files with 4 additions and 420 deletions.
3 changes: 0 additions & 3 deletions src/core/etl/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -672,9 +672,6 @@ $flow->process(new Rows(...))
->run();
```

This function is internally using [filter transformer](src/Flow/ETL/Transformer/FilterRowsTransformer.php).


## Delayed Execution

Reading from the source, transforming data, even loading to sink is executed only by one of the following
Expand Down
11 changes: 1 addition & 10 deletions src/core/etl/docs/complex_transformers.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,16 +6,7 @@ Explanation of complex transformers and their arguments:

### Complex Transformers

Below transformers might not be self descriptive and might require some additional options/dependencies.

#### Transformer - FilterRows

Available Filters

- [all](../src/Flow/ETL/Transformer/Filter/Filter/All.php)
- [any](../src/Flow/ETL/Transformer/Filter/Filter/Any.php)
- [callback](../src/Flow/ETL/Transformer/Filter/Filter/Callback.php)
- [opposite](../src/Flow/ETL/Transformer/Filter/Filter/Opposite.php)
Below transformers might not be self-descriptive and might require some additional options/dependencies.

#### Transformer - EntryNameStyleConverter

Expand Down
14 changes: 2 additions & 12 deletions src/core/etl/src/Flow/ETL/DataFrame.php
Original file line number Diff line number Diff line change
Expand Up @@ -31,8 +31,6 @@
use Flow\ETL\Transformer\DropDuplicatesTransformer;
use Flow\ETL\Transformer\EntryExpressionEvalTransformer;
use Flow\ETL\Transformer\EntryExpressionFilterTransformer;
use Flow\ETL\Transformer\Filter\Filter\Callback;
use Flow\ETL\Transformer\FilterRowsTransformer;
use Flow\ETL\Transformer\JoinEachRowsTransformer;
use Flow\ETL\Transformer\JoinRowsTransformer;
use Flow\ETL\Transformer\KeepEntriesTransformer;
Expand Down Expand Up @@ -197,18 +195,10 @@ public function fetch(?int $limit = null) : Rows

/**
* @lazy
*
* @param callable(Row $row) : bool|Reference\Expression $callback
*/
public function filter(callable|Reference\Expression $callback) : self
public function filter(Reference\Expression $callback) : self
{
if ($callback instanceof Reference\Expression) {
$this->pipeline->add(new EntryExpressionFilterTransformer($callback));
}

if (\is_callable($callback)) {
$this->pipeline->add(new FilterRowsTransformer(new Callback($callback)));
}
$this->pipeline->add(new EntryExpressionFilterTransformer($callback));

return $this;
}
Expand Down
8 changes: 0 additions & 8 deletions src/core/etl/src/Flow/ETL/Row.php
Original file line number Diff line number Diff line change
Expand Up @@ -60,14 +60,6 @@ public function entries() : Entries
return $this->entries;
}

/**
* @param callable(Entry) : bool $callable
*/
public function filter(callable $callable) : self
{
return new self($this->entries->filter($callable));
}

/**
* @throws InvalidArgumentException
*/
Expand Down
16 changes: 0 additions & 16 deletions src/core/etl/src/Flow/ETL/Row/Entries.php
Original file line number Diff line number Diff line change
Expand Up @@ -90,22 +90,6 @@ public function count() : int
return \count($this->entries);
}

/**
* @param callable(Entry) : bool $callable
*/
public function filter(callable $callable) : self
{
$entries = [];

foreach ($this->entries as $entry) {
if ($callable($entry)) {
$entries[$entry->name()] = $entry;
}
}

return self::recreate($entries);
}

/**
* @throws InvalidArgumentException
*/
Expand Down
18 changes: 0 additions & 18 deletions src/core/etl/src/Flow/ETL/Transformer/Filter/Filter.php

This file was deleted.

47 changes: 0 additions & 47 deletions src/core/etl/src/Flow/ETL/Transformer/Filter/Filter/All.php

This file was deleted.

47 changes: 0 additions & 47 deletions src/core/etl/src/Flow/ETL/Transformer/Filter/Filter/Any.php

This file was deleted.

44 changes: 0 additions & 44 deletions src/core/etl/src/Flow/ETL/Transformer/Filter/Filter/Callback.php

This file was deleted.

35 changes: 0 additions & 35 deletions src/core/etl/src/Flow/ETL/Transformer/Filter/Filter/Opposite.php

This file was deleted.

54 changes: 0 additions & 54 deletions src/core/etl/src/Flow/ETL/Transformer/FilterRowsTransformer.php

This file was deleted.

2 changes: 1 addition & 1 deletion src/core/etl/tests/Flow/ETL/Tests/Unit/DataFrameTest.php
Original file line number Diff line number Diff line change
Expand Up @@ -577,7 +577,7 @@ public function extract(FlowContext $context) : \Generator
}
}
)
->filter(fn (Row $row) => $row->valueOf('id') % 2 === 0)
->filter(ref('id')->mod(lit(2))->same(lit(0)))
->fetch();

$this->assertCount(5, $rows);
Expand Down
35 changes: 0 additions & 35 deletions src/core/etl/tests/Flow/ETL/Tests/Unit/Filter/AllTest.php

This file was deleted.

Loading

0 comments on commit 61b2a6c

Please sign in to comment.